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COMPUTER SYSTEM WITH TRANSPARENT 
DATA MIGRATION BETWEEN STORAGE 
VOLUMES 

BACKGROUND OF THE INVENTION 

The present invention relates to the management and 
maintenance of large computer systems and particularly to 
automated methods and apparatus for the movement of data 
(migration) from location in the system to another. 

In 1983, International Business Machines Corporation of 
Armonk, N.Y (IBM) described the requirements and capa- 
bilities needed in order to efficiently manage and maintain 
the storage in the modem large data center and to evolve 
toward automated storage management IBM's Data Facility/ 
System Management Storage (DF/SMS) capabilities. In 
discussing device support and exploitation, IBM identified 
the requirement for the timely support and exploitation of 
new device technologies and the need for tools to simplify 
the movement of data (migration) to new subsystems. 

A standard set of tools has been provided by IBM and 
other software developers that has allowed data to be 
automatically copied, archived, and restored. Evolutionary 
improvements have been made to these tools in the areas of 
performance, usability, and in some cases data availability; 
but a problem still exists in that the availability capabilities 
of these facilities have not kept pace with the availability 
requirements that exist in data centers. The storage admin- 
istrator must be able to support the increasing demands of 
continuous 24 hour by 7day data availability. 

There is an explosive growth in the need to store and have 
on-demand access to greater and greater pools of data. As 
capacity requirements skyrocket, data availability demands 
increase. These factors coupled with the need to control 
costs dictate that new RAID (Redundant Array of Indepen- 
dent Disks) storage technology be implemented. The 
dilemma faced by data center management is that the 
implementation of new storage technology is extremely 
disruptive and therefore conflicts with the need to maximize 
availability of the data. Therefore, an additional tool is 
required that will allow data to be nondisruptively relocated 
or migrated within the data center. 

Essentially, a data migration facility provides the ability to 
"relocate" data from one device to another device. A logical 
relationship is defined between a device (the source) and 
another device (the target). The logical relationship between 
a source and target volume provides the framework for a 
migration. The data migration facility controls multiple 
concurrent migrations in a single group that is called a 
session. A migration is the process that causes the data on the 
source to be copied to the target. 

The characteristics that are critical components of a 
transparent data migration facility include the following: 
The facility is completely transparent to the end user and 
the application program. No application disruption is 
required in order to make program changes and the 
facility is dynamically and nondisruptively activated. 
The facility provides for full data access during the data 
migration. The data on the source volume is available 
to the end user for read and write access. 
The facility provides for a dynamic and nondisruptive 
takeover of the target volume, when the source and 
target volumes are synchronized. 
The migration facility must ensure complete data integ- 
rity. 

The migration facility should NOT be restricted to any 
control unit model type or device type. All devices in 
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the data center should be able to participate in a 
migration and the facility should support a multiple 
vendor environment. 
The State of the Industry 
5 Migration facilities that exist today were primarily 
designed for disaster recovery or the facilities were meant to 
address single volume failures. However, these facilities can 
also be used as data migration tools. These facilities differ in 
implementation and use a combination of host software 
io and/or control unit firmware and hardware in order to 
provide the foundation for the data migration capability. 
Local Mirroring 

The IBM 3990 host extended feature IBM Dual Copy and 
the EMC Symmetrix, from EMC Corporation (EMC), mir- 

15 roring feature are two examples of local mirrors. A source 
volume and a target volume are identified as a mirrored 
paired and at the creation of the mirrored pair, data is 
transparently copied or migrated to the secondary volume. 
Continuous read and write access by applications is allowed 

20 during the data migration process and all data updates are 
reflected to the secondary volume. 

In the case of the IBM 3990 host, the mirror is under the 
complete control of the system operator. For example, 
through the use of system commands a mirrored pair can be 

25 created. At create time, the data will be copied to a second- 
ary device. At the completion of this copy, the operator can 
then disconnect the pair and assign the secondary device to 
be the primary. This is called Transient Dual Copy and is an 
example of Dual Copy being used as a migration facility. 

30 The function of the EMC mirroring feature is to maximize 
data availability. The EMC subsystem will disconnect the 
mirror in the event of a failure of one of the paired devices. 
The mirror will be automatically reconstructed when the 
failing device is repaired by EMC field engineers. Unlike 

35 Dual Copy, the EMC subsystem does not provide an external 
control mechanism to enable the user to selectively initiate 
and disconnect mirrored pairs. Therefore, the EMC mirror- 
ing feature can not be used as a migration facility. 

Standard mirroring has a major restriction that prevents its 

40 universal utilization as a transparent data migration tool. The 
source and target volumes must be attached to the same 
logical control unit and therefore data can only be relocated 
within a single control unit. Although limited, this capability 
is an important tool to the storage administrator. 

45 Remote Mirroring 

IBM 3990-6 and EMC Symmetrix features support 
remote mirroring. A remote mirror function exists when 
paired devices can exist on different control units and 
subsystems. The primary objective of this function is to 

50 provide a disaster recovery capability. However, a remote 
mirroring function can also be used as a data migrator. 

DF/SMS extended Remote Copy (XRC) is a host-assisted 
remote mirror method that uses components of DF/SMS and 
DFP (Data Facility Product). The major component is the 

55 System Data Mover (SDM). This component is also used for 
Concurrent Copy. An IBM 3990-6 (or compatible) host is 
required as the primary or sending control unit. The sec- 
ondary or receiving control unit can be an IBM 3990-3 or -6 
or compatible host. 

60 Other characteristics of XRC include: 

Host application response time is not impacted because 
updates are reflected on the secondary volume asyn- 
chronously. The application does not have to wait for 
the new data to be copied to the secondary volume. The 

65 SDM reads data from the IBM 3990-6 "sidefile" and 
records the update on log files and then writes to the 
remote mirror on the recovery control unit. 
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A common timer is required to insure updates are pro- 
cessed in the correct order and therefore target data 
integrity is guaranteed in a multiple system environ- 
ment. 

No dynamic takeover is supported. Intervention is 

required in order to utilize the secondary copy. 
To invoke XRC as a data migration facility, the following 
steps are required. After identification of the source and 
target pair of devices, an image copy begins and the session 
is placed in a "duplex pending" state. All users of the source 
volume have total read and write access to the data. Source 
updates are reflected on to the target volume. When the copy 
is complete, the session enters the "duplex" state. The 
operator must query the pair in order to determine this state 
change. At this time, all applications using the source 
volume must be brought down in order to synchronize the 
source and target devices. Final synchronization is deter- 
mined by operator command (XQUERY). This query dis- 
plays a timestamp that represents the time of last update so 
that the operator can be assured that all updates have been 
reflected on the target. 

Although XRC does have some of the requirements for 
transparent migration, XRC does not have all of them. 
XRC is transparent to the end user and the application 
program. No application program changes are required 
and the facility is dynamically activated. 
The data on the source volume is available for read and 

write access during the XRC migration. 
XRC causes a disruption because XRC does NOT support 
a dynamic and nondisruptive takeover of the target 
volume when the source and target volumes are syn- 
chronized. The impact of this disruption can be expen- 
sive. All applications with data resident on the source 
-.volume must be disabled during the takeover process. 
XRC ensures complete data integrity through the use of 

the journaling data sets and a common timer. 
XRC is a relatively "open" facility and therefore supports 
a multiple vendor environment. Any vendor that sup- 
ports the IBM 3990-6 XRC specification can partici- 
pate as the sending or receiving control unit in an XRC 
session. Any vendor that supports the IBM 3990-3 or 
basic mode IBM 3990-6 specification can participate as 
the receiving control unit in an XRC session. 
XRC is complex to use and therefore is operationally 

expensive and resource intensive. 
The IBM 3990-6 host also supports a feature that is called 
Peer-to-Peer Remote Copy (PPRC). PPRC is host indepen- 
dent and therefore differs from XRC in several ways. First, 
there is a direct ESCON (Enterprise Systems Connection) 
fiber link from one IBM 3990-6 host to another IBM 3990-6 
host. With this fiber link connection, the primary IBM 3990 
host can directly initiate an I/O Input/Output operation to the 
secondary IBM 3990 host. Secondly, PPRC operates as a 
synchronous process which means that the MVS (Multiple 
Virtual Systems) host is not informed of I/O completion of 
a write operation until both the primary and secondary IBM 
3990 host control units have acknowledged that the write 
has been processed. Although this operation is a cache-to- 
cache transfer, there is a performance impact which repre- 
sents a major differentiator of PPRC over XRC. The service 
time to the user on write operations for PPRC is elongated 
by the time required to send and acknowledge the I/O to the 
secondary IBM 3990 host. 

The link between the IBM 3990 host controllers utilize 
standard ESCON fiber but does require an IBM proprietary 
protocol for this cross controller communication. This pro- 



5,066 

4 

prietary link restricts the use of PPRC to real IBM 3990-6 
host controllers only and therefore does not support a 
multiple vendor environment. 

As suggested above, PPRC can also be used as a migra- 

5 tion facility. PPRC requires a series of commands to be 
issued to initiate and control the migration and is therefore 
resource intensive. IBM has a marketing tool called the 
PPRC Migration Manager that is used to streamline a 
migration process with the use of ISPF (Interactive Struc- 

10 tured Program Facility) panels and REXX (Restructured 
Extended Executor) execs. 

A migration using PPRC (Release 1) does not support an 
automatic takeover to the secondary device. In March of 
1996, IBM announced an enhancement to PPRC called 

15 P/DAS, PPRC Dynamic Address Switch, which apparently 
when available eliminates the requirement to bring down the 
applications in order to perform the takeover of the target 
device. Therefore, P/DAS may allow I/O to be dynamically 
redirected to the target volume when all source data has been 

20 copied to that device. 

Use of P/DAS is restricted to IBM 3990-6 controllers and 
is supported only in an MVS/ESA (Multiple Virtual 
Systems/Enterprise Systems Architecture) 5.1 and DFSMS/ 
MVS 1.2 environment. Therefore the enhancement offered 

25 by P/DAS is achieved at the cost of prerequisite software. 
Furthermore, the dynamic switch capability is based on the 
PPRC platform and therefore supports only a IBM 3990-6 
environment. 

Although PPRC does have some of the requirements for 
30 transparent migration, PPRC does not have all of them. 
PPRC is transparent to the end user and the application 
program. No application program changes are required 
and the facility is dynamically activated. 
The data on the source volume is available for read and 
35 write access during a PPRC migration. 

P/DAS apparently supports a dynamic and nondisruptive 
takeover of the target volume when the source and 
target volumes are synchronized. 
^ PPRC ensures complete data integrity because a write 
operation will not be signaled complete until the pri- 
mary and the secondary IBM 3990 control units have 
acknowledged the update request. This methodology 
will elongate the time required to perform update 
45 operations. 

PPRC requires a proprietary link between two control 
units manufactured by the same vendor. For example, 
only IBM 3990-6 control units can participate in an 
IBM PPRC migration. Therefore PPRC does NOT 
50 support a multiple vendor environment. 

PPRC is complex to use and therefore is operationally 

expensive and resource intensive. 
EMC Corporation's remote mirroring facility is called 
Symmetrix Remote Data Facility (SRDF). The SRDF link is 
55 proprietary and therefore can only be used to connect dual 
Symmetrix 5000 subsystems. 

SRDF has two modes of operation. The first is a PPRC- 
like synchronous mode of operation and the second is a 
"semi-synchronous" mode of operation. The semi- 
60 synchronous mode is meant to address the performance 
impact of the synchronous process. In the synchronous 
mode, the host is signaled that the operation is complete only 
when both the primary and the secondary controllers have 
acknowledged a successful I/O operation. In the semi- 
65 synchronous mode, the host is signaled that the operation is 
complete when the primary controller has successfully com- 
pleted the I/O operation. The secondary controller will be 
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sent the update asynchronously by the primary controller. 
No additional requests will be accepted by the primary 
controller for this volume until the secondary controller has 
acknowledged a successful I/O operation. Therefore in the 
SRDF semi-synchronous mode, there may one outstanding 5 
request for each volume pair in the subsystem. 

EMC personnel must be involved in all SRDF activities 
unless the user has installed specialized host software that is 
supplied by EMC. The proprietary nature of SRDF restricts 
its use as a data migration facility. The primary function of 10 
SRDF is to provide data protection in the event of a disaster. 

Late in 1995, EMC announced a migration capability that 
is based on the SRDF platform. This facility allows a 
Symmetrix 5000 to directly connect to another vendor's 
subsystem. The objective of the Symmetrix Migration Data 15 
Service (SMDS) is to ease the implementation of an EMC 
subsystem and is not meant to be a general purpose facility. 
SMDS has been called the "data sucker" because it directly 
reads data off another control unit. The data migration must 
include all of the volumes on a source subsystem and the 20 
target is restricted to a Symmetrix 5000. 

An EMC Series 5000 subsystem is configured so that it 
can emulate the address and control unit type and device 
types of a existing subsystem (the source). This source 
subsystem is then disconnected from the host and attached 25 
directly to the 5000. The 5000 is then attached to the host 
processor. This setup is disruptive and therefore does cause 
an application outage. 

The migration begins when a background copy of the 
source data is initiated by the 5000 subsystem. Applications 30 
are enabled and users have read and write access to data. 
When the target subsystem (the 5000) receives a read 
request from the host, the data is directly returned if it has 
already been migrated. If the requested data has not been 
migrated, the 5000 will immediately retrieve the data from 35 
the source device. When the target subsystem receives a 
write request, the update is placed only on the 5000 and is 
not reflected onto the source subsystem. This operation 
means that updates will cause the source and target volumes 
to be out of synchronization. This operation is a potential 40 
data integrity exposure because a catastrophic interruption in 
the migration process will cause a loss of data for any 
volume in the source subsystem that has been updated. 

Although the Symmetrix Migration Data Service does 
have some of the requirements for transparent migration, 45 
SMDS does not have all of them. 

a. SMDS is not transparent to the end user and the 
application program. Although no application program 
changes are required, the facility cannot be nondisrup- 
tively activated. All applications are deactivated so that 50 
the 5000 can be installed and attached to the host in 
place of the source subsystem. Specialized software is 
loaded into the Symmetrix 5000 to allow it to emulate 
the source subsystem and initiate the data migration. 
This disruption can last as long as an hour. 55 

1 . The data on the source volume is available for read 
and write access during a SMDS migration. 

2. SMDS may support a dynamic and nondisruptive 
takeover of the target volume when the source and 
target volumes are synchronized. At the end of the 60 
migration, the source subsystem must be discon- 
nected and the migration software must be disabled 
and it is unknown whether this is disruptive and an 
outage is required. 

3. SMDS can link to control units manufactured by 65 
other vendors. However, the purpose of SMDS is to 
ease the disruption and simplify the installation of an 
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EMC 5000 subsystem. Data can only be migrated to 
an EMC subsystem. Therefore SMDS does NOT 
support a multiple vendor environment. 
SMDS does NOT ensure complete data integrity. 
During the migration, data is updated on the target 
subsystem and is not reflected on the source 
subsystem. A catastrophic error during the migra- 
tion can cause the loss of all application updates. 
The State of the Art in a Summary 
The products that are available on the market today do not 
meet all of the data migration requirements. 

The maintenance of continuous data availability is a 
fundamental mission of data centers. In order to support this 
goal, the migration facility must be initiated transparent to 
all applications and provide a means for the nondisruptive 
takeover of the target device. 

The value of data and information is critical to the 
operation and competitiveness of the enterprise and there- 
fore any exposure to possible data loss is unacceptable. 

The control of the costs of the ever expanding storage 
assets is a large component in financing an information 
technology infrastructure. A competitive multiple vendor 
environment provides a mechanism to support effective cost 
management. Therefore, the migration facility should be 
vendor independent. Accordingly, there is a need for 
improved data migration methods and apparatus which have 
all the data migration requirements. 

SUMMARY OF THE INVENTION 

The present invention is a data migration facility for 
managing and maintaining large computer systems and 
particularly for automatic movement of large amounts of 
data (migration of data) from one data storage location to 
another data storage location in the computer system. 

The computer system has a plurality of storage volumes 
for storing data used in the computer system, one or more 
storage control units for controlling I/O transfers of data in 
the computer system from and to the storage volumes, one 
or more application programs for execution in the computer 
system using data accessed from and to the storage volumes, 
one or more operating system programs for execution in the 
computer system for controlling the storage volumes, the 
storage control units and the application programs, and a 
data migration program for migrating data from one of the 
data volumes designated as a source volume to one of said 
data volumes designated a target volume while the applica- 
tion programs are executing using data accessed from and to 
the storage volumes. 

The data migration program includes a main module to 
control the start of a migration session when the application 
programs are using data accessed to and from the source 
volume, to migrate data from the source volume to the target 
volume, and to end the migration session whereby the 
application programs are using data accessed to and from the 
target volume The data migration program includes a vol- 
ume module to control the volumes during the migration 
session. The data migration program includes a copy module 
to control the copying of data from the source module to the 
target module during the migration session. The data migra- 
tion program includes a monitor module for monitoring I/O 
transfers to the data volumes during the migration sessions. 

In one embodiment, the data migration program includes 
dynamic activation and termination, includes non-disruptive 
automatic swap processing that does not require operator 
intervention, is transparent to applications and end-users 
while providing complete read and write activity to storage 
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volumes during the data migration, permits multiple migra- such as the MVS operating system. The operating systems 

lions per session, permits multiple sessions. Additionally, control the operation of the host computers and the execu- 

the installation is non-disruptive (a computer program that tion of a number of customer applications 2 and a TDMF 

can execute as a batch process rather than requiring the application. The TOMS application is used for migrating 

addition of a new hardware sub-system to the computer 5 data from one location in the enterprise system to another 

system), requires no IPL (Initial Program Load) of the location in the enterprise system. The data locations in the 

operating system, is vendor independent with any-to-any enterprise system 1 are located on storage volumes 5. Data 

device migration independent of DASD (Direct Access is transferred to and from the storage volumes 5 under 

Storage Device) control unit model type or device type. The control of the storage control units 4. The architecture of the 

data migration program includes a communication data set 10 FIG. 1 system is well-known as represented, for example, by 

(COMMDS) located outside the DASD control unit which Amdahl Corporation of Sunnyvale, Calif. (Amdahl) and 

helps ensure vendor independence. IBM mainframe systems. 

The data migration facility has complete data integrity at For efficient operation of the enterprise system 1, it is 

all times, provides the ability to introduce new storage necessary at times to migrate data from one of the storage 

subsystems with minimal disruption of service (install is 15 volumes 5 to another one of the storage volumes 5. The 

disruptive), allows parallel or ESCON channel connections storage volumes 5 between which the migration occurs may 

to ensure vendor independence and can be implemented as be located at the same place under control of a single 

computer software only, without need for dependency on operating system or may be located in volumes under the 

hardware microcode assist. control of different operating systems. Also, some of the 

The foregoing and other objects, features and advantages 20 volumes ma Y be located geographically remote such as in 

of the invention will be apparent from the following detailed d ^rent cities, states or countries. When located remotely, 

description in conjunction with the drawings. the remote computers are connected by a high-speed data 

communication line such as a 13 line. 

BRIEF DESCRIPTION OF THE DRAWINGS The TDMF migration is intended for flexibility, for 

prr 1 A*™vto -> uwi. a- -p * * 25 example, so that multiple MVS/ESA releases are supported 

system ^ ^ ^ < 4 ' 2 ' 43 > ^ 5 ' 2 ' OS ^ 90 VL1 > VL2 > V1 ' 3 > and V2 4 )> 50 

' . that shared data system environments are supported, so that 

FIG. 2 depicts further detads of the enterprise system of C KD/E (Count Key Data/Extended) compatible 388x and 

IG * 399x control units are supported (Read Track CCW 

FIG. 3 depicts a single MVS operating system embodi- 30 (Channel Control Word) is required in the disclosed 

ment for migrating data from a source volume to a target embodiment), so that 3380 and 3390 device geometries are 

volume. supported, so that flexible device pairing options are pos- 

FIG. 4 depicts a multiple MVS operating system embodi- sibie ( uses device pairs with equal track sizes and numbers 

ment for migrating data from a source volume to a target °f cylinders and requires the target volume to be equal to or 

volume. 35 greater than the source volume), so that a single TDMF 

FIG. 5 depicts a flow diagram of the different states of the session can support U P to 640 oancunent migrations, so that 
migration process a sm S le TDMF session can support concurrent migrations 

FIG. 6 depicts a block diagram of the different software wi ! h 6i f eliaS , ™ bo1 Uni ' and ^'If S ' and 50 * at 

components that control the master slave operation used to m T ca ? abl ^ ™lable. 

perform the migration process. 40 The TDMF migration is intended to have an ease of use 

tj,^ - . • . , ,. „ . , that includes, for example, easy installation, a simple param- 

7 f^T \ 8ram the T SUb " teSk eter driven P rocess > a centralized control and monitoring 

components that control the copying of data in the migration capab ji itv> and me abaitv t0 have integrated online help J 

process. documentation. 

FIG. 8 depicts a block diagram of the customer I/O 45 The TDMF migration is intended to have a minimal 

request components that control the operation of I/O impact on performance featuring, for example, a multiple 

requests during the migration process. tasking design, efficient user I/O scanning, asynchronous 

FIG. 9 depicts a block diagram of the interrelationship of copy and refresh processes, minimization of device logical 

the different software components used to perform the quiesce time and integrated performance measurement capa- 

migration process. 50 bility. 

FIG. 10 depicts the source and target volumes before a The TDMF migration is intended to have application 

TDMF migration. transparency including, for example, dynamic installation 

FIG. 11 depicts the source and target volumes after a and termination, no requirement for application changes, 

TDMF migration. continuous read and write access, dynamic enablement of 

55 MVS intercepts and dynamic and nondisruptive takeover of 

DETAILED DESCRIPTION target devices. 
Enterprise System— FIG. 1 The TDMF migration is intended to have data integrity 

The Transparent Data Migration Facilities (TDMF) is including, for example, continuous heartbeat monitoring, 
resident as an application program under the MVS/ESA host dynamic error detection and recovery capability, passive and 
operating system of the enterprise system 1 of FIG. 1. The 60 nondestructive I/O monitoring design, optional data corn- 
Transparent Data Migration Facilities (TDMF) is an appii- pare capability and maintenance of audit trails, 
cation program among other application programs 2. The Multiple Operating System Environment — FIG. 2 
TDMF migration provides model independence, operational In FIG. 2, the multiple operating systems 3 include the 
flexibility, and a completely transparent migration. MVS operating systems 3-1, 3-2, . . . , 3-M. These operating 

In FIG. 1, the enterprise system 1 is a conventional 65 systems 3 are typically running on a plurality of different 

large-scale computer system which includes one or more host computers with one or more operating systems per host 

host computer systems and one or more operating systems computer. Each of the operating systems 3-1, 3-2, . . . , 3-M 
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is associated with a plurality of application programs inchid- Specifically, the storage control units 4-1, 4-2, ... , 4-M 

ing the application programs 2-1, 2-2, . . . , 2-M, respec- control accesses to and from the volumes 5-1, 5-2, . . . , 5-M, 

tively. The application programs 2-1, 2-2, . . . , 2-M are respectively. Specifically, the volumes 5-1 include the vol- 

conventional application programs as would be found in a umes 5-1^ ... , 5-1 w » the volumes 5-2 include the volumes 
customer enterprise system. In addition to the conventional 5 5-2 l9 . . . , 5-2 w , . . . , and the volumes 5-M include the 

application programs, the TDMF application program 2-T is volumes 5-M lf . . . , 5-M^,, respectively, 

also present on each of the operating systems 3-1, At some point in time it becomes desirable in the enter- 

3- 2, . . . , 3-M that are to be involved in a data migration. prise system of FIG. 4 to migrate data from a source volume 
In FIG. 2, each of the operating systems 3-1, 3-2, . . . , 3-M VOLy to a target volume VOLy. In the example of FIG. 4, 

is associated with a storage control unit complex 4 including ao the source VOLy is designated as 5-l a controlled through 
the storage control unit complexes 4-1, 4-2, . . . , 4-M, the SCU^ 4-1 and the target volume VOL y is designated 
respectively. 5-M a and is controlled through the SCU y designated 4-M. 

Each of 'the storage control units 4-1, 4-2, ... , 4-M is The data migration occurs from the source VOL^ to the 
associated with one or more volumes. In the example shown, target VOL^ without interruption of the customer applica- 
the storage control unit 4-1 is associated with the data is tions 2-1, 2-2, . . . , 2-M. 

storage volumes 5-1^ . . . , 5-1^. Similarly, the storage In order to control the data migration from the source 
control unit 4-2 is associated with the volumes 5-2 lf 5-2^. VOLy to the target VOLy, the TDMF application 2-T 
Finally, the storage control unit 4-M is associated with the operates in a distributive manner across each of the operat- 
volumes 5-M lf . . . , S-M VM . ing systems 3-1, 3-2, . . . , 3-M. Specifically, the operation 

Hie operation of the FIG. 2 data storage is conventional 20 of the TDMF migration application is in the form of a 
in the MVS environment. Specifically, any one of the master/slave implementation. Each of the operating systems 
applications 2-1, 2-2, . . . , 2-M may store or retrieve data 3-1, 3-2, . . . , 3-M includes a corresponding instance of the 
from any one of the volumes 5 under control of the operating TDMF application, namely, the TDMF application 2-T. One 
systems 3-1, 3-2, . . . , 3-M through the storage control units of the applications is designated as a master TDMF appli- 

4- 1 4-2, , 4-M. 25 cation and the other of the applications is designated as a 
At any time', any one of the volumes 5 may be designated slave TDMF application. In the example of FIG. 4, the 

for replacement or otherwise become unavailable so that the TDMF application associated with the MVS 3-1 is desig- 
data on the volume must be moved to one of the other nated as the master TDMF application 2-T mas . Each of the 
volumesofFIG. 2. For example, the data of a source volume other operating systems 3-2, . . . , 3-M is associated with a 
X may be migrated to a target volume Y. In FIG. 2, by way 30 slave application 2-T sll , . . . , 2-T, w , respectively, 
of example, volume 5-l 3 has been designated as the source The data migration in the FIG. 3 and FIG. 4 systems is 
volume and volume 5-M t has been designated as the target carried out with a TDMF application 2-T (with a separate 
volume. The objective is to move all the data from the source instance of that application for each MVS operating system) 
volume X to the target volume Y transparently to the without necessity of any modified hardware connections, 
operation of all of the applications 2-1, 2-2, . . . , 2-M so that 35 TDMF Migration Stages — FIG. 5 

the enterprise system of FIG. 2 continues operating on a In FIG. 5, the relationship between the migration stages 
continuous 24-hour by seven day data availability without for migrating data between a source volume and a target 
significant disruption. volume is shown. Specifically, the migration commences 

Single MVS Operating System Environment— FIG. 3 with an IMTIAUZAnON/ACnVATION phase 10, fol- 

In FIG. 3, the single master MVS operating system 3-1 40 lowed by a COPY phase 11, followed by a REFRESH phase 
controls the operation of the applications 2-1 on a source 12, followed by a QUIESCE phase 13, followed by a 
volume 5y At some time, the enterprise system requires that SYNCHRONIZE phase 14, followed by a REDIRECT 
the source volume 5^ be taken out of service requiring the phase 15, followed by a RESUME phase 16, and ending in 
data on the source volume 5 X to be migrated to a target a TERMINATION phase 17. If during the 
volume 5 y . The operating system 3-1 controls transfers to 45 INITIALIZATION/ACTIVATION phase 10, an error 
and from the source volume 5^ through the control unit 4 X . occurs, the error is detected by the ERROR module 20 
Similarly, the MVS operating system 3-1 controls transfers which passes the flow to the TERMINATION stage 17. If an 
to and from the target volume 5 y through the storage control error is detected during the COPY phase 11, the ERROR 
unit 4y, The data migration from the source volume 5 X to the module 21 passes the flow to the TERMINATION stage 17. 
target volume 5 Y is under control of the TDMF application 50 If an error is detected during the REFRESH phase 12, the 

2- T r In order to achieve the transparent data migration from ERROR module 22 passes the flow to the TERMINATION 
the source volume 5^- to the target volume 5 y , an additional phase 17. 

data volume 5-1 which is not involved in the data migration If an error occurs during the QUIESCE phase 13 or the 

is available to the MVS operating system 3-1. During the SYNCHRONIZE phase 14, the ERROR modules 23 and 24, 
entire migration from the source volume 5 X to the target 55 respectively, pass the flow to the RESUME phase 16. If an 
volume 5 Y , the applications 2-1 continue their operation error occurs during the REDIRECT phase 15, the ERROR 
without disruption and while maintaining complete data module 25 passes the flow to the BACKOUT module 26 
integrity. which then returns the flow to the RESUME phase 16. 

Multiple Operating System Environment Example — FIG. 4 The migration phases of FIG. 5 are active in each of the 

In FIG. 4, the MVS operating systems 3-1, 3-2, . . . , 3-M 60 MVS operating systems 3 and the MASTER TDMF appli- 
are all part of the enterprise system 1 of FIG. 1. Each of the cation and the MASTER MVS operating system insure that 
operating systems 3-1, 3-2, . . . , 3-M is associated with one phase is completed for all operating systems, both 
corresponding application programs 2-1, 2-2, . . . , 2-M. master and all slaves, before the next phase is entered. 
Each of the applications 2-1, 2-2, . . . , 2-M is operative MASTER/SLAVE System— FIG. 6 

under the control of the MVS operating systems 3-1, 65 The MASTER/SLAVE system operation is under control 

3- 2, . . . , 3-M to access the storage volumes 5 through the of the TDMF application 2-T. That application, and each 
storage control units 4-1, 4-2, . . . , 4-M. instance thereof in an MVS operating system includes the 
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modules of FIG. 6. Specifically, the TDMFMAIN module is located on the source volume will be reflected on the target 
the main task which calls other ones of the modules for volume. The TDMFIMON module only monitors volumes 
controlling the phasewise execution of the migration as set involved in a migration session of FIG, 5 and any I/O 
forth in FIG. 5. The TDMFMAIN module starts the TDMF operations to volumes not involved in the TDMF migration 
session of FIG. 5 on the master and slave systems, opens all 5 session are not impacted in any fashion, 
necessary files, reads and validates control cards and vali- TDMF Module Overall Architecture — FIG. 9 
dates all parameters and volumes to be involved in the j n jqc. 9, the overall architecture of the computer soft- 
migration session. ware for implementing the TDMF migration of FIG. 5 is 

The TDMFVOL module controls the migration process of shown. The TDMFMAIN module functions to control sys- 

any and all volumes being migrated within the TDMF 10 t ems initialization and system determination and calls the 

session of FIG. 5. TDMFVOL module, the TDMFICOM module, and the 

The TDMFICOM generates channel programs to imple- TDMF1VOL module, 

ment I/O operations being requested to the COMMUNICA- ^ TDMFVOL module is responsible for volume ini- 

TIONS DATA SET (COMMDS) via parameters passed by action with the TDMFIVOL module, volume activation 

the caller. The caller may request that the system initialize 15 with the tdmfavOL module, volume quiesce with the 

the COMMDS or to selectively read or write caller specified TOM FQVOL modui ^ yolume resumfi wkh tfae tomj^. 

TDMF control blocks as required by the controller. The VOL module and volume termination with the TDMFTVOL 

TDMFICOM module is called by the TDMFMAIN module mo dule. The TDMFVOL module calls or returns to the 

onl y* TDMFMAIN module, the TDMFICOM module, the TDM- 

The TDMFIVOL module generates channel programs to 20 FIV0L module> md the TDMFCOPY module, 

implement I/C ^operations being requested to volumes being ^ TOMFCO p Y mocMe is called b m6 retums to the 

migrated to the IDMF session of FIG. 5 via parameters TDMFVOL module and the TDMFIVOL module, 
passed by the caller. 

Hie TDMFSIO module issues a request to the MVS ™< L™ MF * C ? M mM ** ^ * I * m * d * 

INPUT/OUTPUT SUPERVISOR (IOS) component to per- 25 TDMFSIO mo dule. 

form the I/O operation represented by the INPUT/OUTPUT The TDMFIVOL module calls and is returned to the 

SUPERVISOR control BLOCK (IOSB) in conjunction with TDMFSIO module. 

its SERVICE REQUEST BLOCK (SRB) as requested by the Details of the various modules of FIG. 6, FIG. 7, FIG. 8 

caller. The IDMFSIO module can be called from the TOM- and FIG. 9 for performing the migration in the phases of 

FICOM module and the TDMFIVOL module. Upon 30 FIG. 5 are described in detail in the following LISTING 1 

completion of the I/O operation requested, control is near the end of the specification before the claims, 

returned to the calling module. Source and Target Volumes Before TDMF Migration — FIG. 

COPY Sub-task— FIG. 7 10 

In FIG. 7, the COPY Sub-task functions with one per In mG j^, a data migration is to occur between the 

volume migration for the master only. The COPY Sub-task 35 volume 5 _ la ^ lhe ^mce through the storage control unit 

includes the TDMFCOPY module which provides the func- SCU X designated 4-1 to the target volume 5-Mj through the 

tions required by the TDMF COPY Sub-task. The TDMF- storage control unit SCU y designated 4-M. The SOURCE 

COPY Sub-task is attached as a Sub-task by the module volume has device address A00, serial number SRC001 and 

TDMFVOL by means of an ATTACHX macro during its ^ ^ the ON-LINE status before the TDMF migration. The 

processing. The TDMFCOPY module implements three 40 target volume has the device address FC0, serial number 

Sub-task phases, namely, the COPY sub-phase, the TGT001 and is in the ON-LINE status before volume 

REFRESH sub-phase, and the SYNCHRONIZATION sub- migration. Before the volume migration begins, all I/O 

phase, operations are being directed to the source volume 5-l 1 . 

^?^^S FIV ° 1 L 1 m0dule may al f be 1 called by the Source and Target Volumes After All TDMF Migration- 

TDMFCOPY module to generate channel programs to 45 ^ 

implement I/O operation being requested to volumes being . " _ _ - , . ^ , , . 

m ,-„ nta j + „ + . Vnn^c o™,vT„ ~f cm c ,rio In FIG. 11, the source volume 5-1 , has a device address 

migrated to the TDMF session of FIG. 5 via parameters _ . . . , , „™ T Txrr . 

assed b the caller ' a ^ number of TGT001 and an OFF-LINE 

tu TnxfrcTn j 1 , ii j u *u TTMuri-¥x/r\T status. The target volume 5-M-. has a device address of FCO, 

The TDMFSIO module when called by the TDMFIVOL . , * c „ nr>riri . 1 , ~ XT T TXTT . 4 , ' 

j i * , . + u uuc iMmTT/nnTnTTT ,~ a senal number of SRC001, and an ON-LINE status. 

module issues a request to the MVS INPUT/OUTPUT 50 ' 

SUPERVISOR (IOS) component to perform the I/O opera- TOMF Migration Operation 

tion represented by the INPUT/OUTPUT SUPERVISOR TDMF is initiated as an MVS batch job. TDMF's job 

CONTROL BLOCK (IOSB) in conjunction with its SER- control language identifies the system type, session control 

VICE REQUEST BLOCK (SRB) as requested by the caller. parameters, and the communications data set (COMMDS). 

Upon completion of the I/O operation requested, control is 55 The session control parameters define the paired source 

returned to the TDMFIVOL module. (VOLy) and target (VOL*) volumes which make up the 

I/O Monitoring FIG. 8 migrations under the control of this session. The control 

In FIG. 8, a block diagram of the monitoring of I/O parameters also define the master system and all slave 

operations during a migration session of FIG. 5 is shown. systems. The COMMDS set is the mechanism that is used to 

The FIG. 8 modules are conventional with the addition of 60 enable all systems to communicate and monitor the health of 

the TDMFIMON module. The purpose of the TDMFIMON the migrations in progress. The COMMDS set is also used 

module is to monitor all customer I/O operations to the as an event log and message repository, 

source and target volumes during the life of an active All systems that have access to the source volume must 

migration session of FIG. 5. The TDMFIMON module have TDMF activated within them. This requirement is 

insures that the primary design objective of insuring data 65 critical in order to guarantee data integrity. One system is 

integrity of the target volume by insuring that any update designated as the master (PARM-MASTER) and all other 

activity by customer I/O operation that changes the data systems are designated as slaves (PARM=SLAVE). 
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The responsibilities of the master system include: chronously from the source volume to the target volume. 

Initialization of the master TDMF environment and the There is an independent COPY sub-task for each active 

COMMDS. migration. 

Validation of the initialization of all slave systems. I/O monitoring provided by the TDMFIMON module of 

The control of all migrations defined in this TDMF FIG. 8 provides the ability to detect updates to the source 

session. volume. A system that detects a source volume update will 

The monitoring of source volume user I/O activity in note this update in a refresh notification record and send this 

order to detect updates. information to the master system. 

The copying of data from the source to the target volume. 10 When a COPY sub-task has completed one pass of the 

Collection of update information from all systems. source volume, the REFRESH phase is initiated by the 

The refresh and synchronization of the target volume due master. During this phase, the updates that have been made 

to detected update activity. t0 tne so UrC e volume are reflected onto the target volume. 

The monitoring of the internal health of the master 15 -j^g REFRESH phase is divided into multiple cycles. A 

environment and the health of all slave systems. refresh cycle is a pass through the entire source volume 

The responsibilities of the slave systems include: processing ail updat es. TDMF measures the time required to 

Initialization of the slave TDMF environment. process ^ refresh operations in a sing i e refresh cycle. The 

Establishment of communication to the master through refresh cydes wiU ^ntmue until the time for a refresh cycle 

the COMMDS. 20 ^ reduced ihai allows the Synchronization Goal to be 

The processing and acknowledgment of all migration ^ vQd ^ a threshold ^ fc ^ Qn ^ collecled 

requests as received from the master. „ , , , , . . , « . 

. . _ . t/^ *• ■* • performance data. When the threshold is reached, this event 

The monitoring of source volume user I/O activity m F . , , i , _ . , 

order to detect updates. sl S nals the master to be read y t0 enter the Qa3BSKX and 

Transmission of update information to the master through 25 Synchronization phases. If there are multiple REFRESH 

the COMMDS. phases during a migration, it is due to the inability of TDMF 

The monitoring of the internal health of this slave envi- to meet the SYNCHRONIZATION goal. This is usually 

ronment and the health of all other systems. because of high write activity on the Source volume. If there 

The master system initiates and controls all migrations. A 3Q are no issues with this, then there is no reason to change the 

migration is broken into the major phases as illustrated in SYNCHRONIZATION goal parameter. 

FIG. 5. The master initiates each phase and all slave systems Prior to the Synchronization phase, a QUIESCE is issued 

must acknowledge in order to proceed. If any system is to the Source volume. In the instance of a multi -system 

unable to acknowledge or an error is detected, the systems migration, the Master issues a request to the Slave(s) to 

will cleanup and terminate the migration. To ensure data 35 Quiesce all I/O to the Source volume (from the slave side), 

integrity, TDMF is designed with a passive monitoring At this time the final group of detected updates are collected 

capability so that all user I/O operations to the source and applied to the Target volume (SYNCHRONIZATION), 

volume will complete as instructed by the application. This At the end of Synchronization, the Master starts the volume 

completion means that if any unexpected event occurs, the ^ REDIRECT (swap) phase. The Target volume has now 

data on the source volume is whole. TDMF will merely become the new Source volume. When all systems have 

cleanup and terminate the migration. verified the Redirect, the Master initiates the RESUME 

Standard cleanup and recovery is active for all phases phase so that user I/O can continue to the new source 

with the exception of the REDIRECT phase. During this volume. The elapsed time between the last QUIESCE phase 

REDIRECT phase, it may be necessary to take additional 45 and the RESUME phase is approximately four (4) seconds 

recovery actions. This specialized recovery is called the plus the ACTUAL SYNCHRONIZATION time (which 

BACKOUT phase. Backout is required due to error detec- should always be less than the specified synchronization 

tion in a multiple system environment. In a multiple system goal). 

migration, the systems may be at differing stages within the The Synchronization Goal default is five (5) seconds. 
REDIRECT phase at the point the error is detected. Some of 50 Synchronization will not occur unless the calculated syn- 

the systems may have completed redirect and others may chronization time is less than the goal. If the synchronization 

have not completed redirect processing. Backout recovery goal is increased, then the time the user I/O is queued 

dynamically determines the state of each system in order to (quiesced) is greater. If the value 999 is used, this equates to 

take the appropriate recovery action. However, even in the 55 synchronize as soon as possible; it does not matter how long 

REDIRECT and BACKOUT phases, the data on the source it takes. This can be a significant amount of time depending 

volume is whole and user data will not be lost. on the write activity of the source volume. Therefore, use 

Referring again to FIG. 5, a TDMF session begins with an discretion when changing this value. 

INITIALIZATION phase, immediately followed by an Application I/O operations during the Copy and Refresh 

ACTIVATION phase. During this phase, all systems confirm 60 phases are impacted no more than if a backup utility is 

the validity of the source and target volumes and enable user backing up the source volume at the same time as the 

I/O activity monitoring. When all systems acknowledge application is executing. The Synchronization goal param- 

successful completion of this process, the master will begin eter may be specified for each volume migration, allowing 

the COPY phase and initiate a sub-task (COPY Sub-task) to 65 the customer to specify the amount of time (in seconds) that 
copy data from the source to the target volume. This COPY he will allow the Synchronization phase to execute. This is 

Sub-task is a background activity and will copy data asyn- the maximum amount of time. 
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Performance Impact by Phase 

The Master and Slave(s) wake up for processing based 
upon a variable which is the minimum number of seconds 
based upon any migration volume's current phase or stage. 
The phases of a volume migration and their associated time 
intervals are: 
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Description 



H:MM:SS.ss Description 



H:MM:SS.ss % 



3380-D No load situation with Compare option on: 
Single Volume 



Copy Time 
Compare Time 
Total Time 



Phase 



Time Interval 



10 



Copy 


30 seconds 


Refresh pass #1 


15 seconds 


Refresh pass #2 


10 seconds 


Refresh pass #3 . . . n 


5 seconds 


Quiesce 


1 second 


Synchronize 


1 second 


(Compare) 


1 second 


Swap/Point-in-Time 


1 second 


Resume 


1 second 


Tenrunation/Completion 


15 seconds 



Copy Time 
Compare Time 
Total Time 



11:31.02 TCBtime 

8:1139 SRB time 
20:11.70 CPU overhead 

3380-K No load situation: 
Single Volume 

TCB time 

SRB time 
34:50.65 CPU overhead 
3390-3 with a heavy write load: 
Single volume 
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30 



35 



This allows TDMF to be responsive with a minimum of 
CPU overhead. 

The CPU overhead associated with running IDMF is less 
than 3 percent on average for the Master system. This is 25 
dependent upon the number of storage volumes within a 
session and write activity against those source volumes. A 
Slave system's CPU overhead will be almost non- 
measurable. 

For example, if the Master job takes 44 minutes, 22 
seconds (2662 seconds total) to migrate 16 volumes, and the 
TCB time is 63.5 seconds, and the SRB time is 2.92 seconds, 
then the CPU overhead is equal to 2.49 percent ((63.5+ 
2.92)/2662) for that session. 

When a multi-volume migration is running in a TDMF 
session, not all volumes are going to be in the same phase 
at the same time. This is because different volumes may have 
different activity against them. The number of channels 
available to each Control Unit (CU) will also be a factor in 
this. Therefore, it is entirely possible to have a 4 volume 
migration running with volume 1 in the copy phase, volume 
2 in a 4th refresh phase, volume 3 completed, and volume 4 
is in the Synchronization phase. 

Placement of the Master 

The Master should be placed on the system that has the 
most updates or on the system where the owning application 
is executing. If multiple TDMF Master sessions are being 50 
run on multiple operating systems, then the MVS system(s) 
must have a global facility like GRS or MIM. This is to 
prevent inadvertent usage of the same volumes in a multi- 
session environment. If a GRS type facility is not available 
in the complex, then all Master sessions must run on the 55 
same operating system. 

Placement of the Communications Dataset 

The Communications Dataset (COMMDS) should be 

placed on a volume with low activity and the volume must 60 

not be involved in any migration pairing. 
Sample Performance Statistics 

In the following tables, time is designated in hours (H), 
minutes (MM), and seconds (SS.ss) and in the form 65 
H:MM:SS.ss; for example, 1:07:20.36 is 1 hour, 7 minutes 
and 20.36 seconds. 



45 



Copy Time 


24:11.05 TCBtime 


Refresh Time 


16:35.23 SRB time 


Sync Time 


6:29.29 


Total Time 


48:25.97 CPU overhead 




3390-3 No load situation: 




8 volume copy 


Copy Time 


TCB time 


Compare Time 


SRB time 


Total Time 


20:24.87 CPU overhead 




3390-3 No load situation: 




16 volume copy 


Copy Time 


TCB time 


Compare Time 


SRB time 


Total Time 


33:52.52 CPU overhead 




3390-3 No load situation: 




32 volume copy 


Copy Time 


TCB time 


Compare Time 


SRB time 


Total Time 


1:07:20.36 CPU overhead 




3390-9 No load situation: 




Single volume 


Copy Time 


1:02:36.67 TCBtime 


Compare Time 


SRB time 



Total Time 



1:03:23.55 CPU overhead 



17.93 
0.33 



3.9S 
0.26 



9.04 
0.72 



30.91 
1.67 



1:04.13 
3.20 



2:13.00 
5.89 



12.54 
0.70 



1.5 



0.2 



0.3 



3.4 



0.3 



Master System Responsibilities 

Initializing the master TDMF environment and the COM- 
MDS. 

Starting and controlling the migrations for all TDMF 
systems. 

Monitoring source volume user I/O activity to detect 
updates. 

Copying data from the source to the target volume. 
Processing detected updates from all systems. 
Performing refresh operations to the target volume to 

reflect update activity. 
Checking the internal health of the master environment 

and the health of all slave systems. 
Slave System Responsibilities 

Initializing the slave TDMF environment and establishing 
communications to the master, using the COMMDS. 

Acknowledging and processing migration requests from 
the master. 

Monitoring source volume user I/O activity and detecting 
updates. 

Notifying the master of update activity through the COM- 
MDS. 

Checking the internal health of the slave environment and 
the health of all systems. 
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At this time, the master issues the quiesce request to all 
systems so that activity to the source volume will be 
inhibited. When the quiesce request is received by a slave 
system, this is the indication for the slave system to enter the 
QUIESCE phase and to quiesce and send to the master the 5 
final group of detected updates so that the master may 
perform the synchronization process or optionally S restart 
an additional REFRESH phase- 
When all systems have successfully acknowledged the 
quiesce request, synchronization will begin in the SYN- 10 
CHRONIZATION phase. The device quiesce is transparent 
to all applications. An application can issue I/O requests; 
however, they will not be executed and will be queued until 
the RESUME phase. The purpose of the collection of 
performance information during the REFRESH phase is to 15 
minimize the device quiesce time. At the conclusion of the 
SYNCHRONIZATION phase, the master will initiate the 
REDIRECT phase. 

The purpose of the REDIRECT phase is to cause I/O 
activity to be redirected to the target volume so that it can 
become the primary device. The master will request that all 
systems perform and confirm a successful redirect before the 
redirect operation is performed by the Master system. When 
all systems have acknowledged this activity, the master will ^ 
then initiate the RESUME phase request so that any queued 
user I/O will begin execution. Subsequent I/O operations 
will then be directed to the target device. 

After all systems have processed the resume request, I/O 
monitoring will be disabled and the migration will terminate 30 
in the TERMINATE phase. 

The TDMF facility simplifies the relocation of user- 
specified data from existing hardware to new storage 
subsystems, for example, without loss of access to data, 
without the imposition of unfamiliar operator procedures, 35 
and without dependence upon vendor firmware or hardware 
release levels. 



18 

TDMF is transparent to the end user and the application 
program. No application program changes are required 
and the facility is dynamically activated. 

TDMF provides for full data access during the data 
migration. The data on the source volume is available 
to the end user for read and write access. 

TDMF supports a dynamic and nondisruptive takeover of 
the target volume when the source and target volumes 
are synchronized. All applications with data resident on 
the source volume may remain enabled during the 
takeover process. 

TDMF ensures complete data in integrity through the use 
of passive I/O monitoring and background copy opera- 
tions. 

TDMF is an "open" facility and therefore supports a 
multiple vendor environment. Any vendor that supports 
the IBM 3880 or IBM 3990 ECKD specification can 
participate as the sending or receiving control unit in a 
TDMF migration. (Read Track CCW support is 
required). 

TDMF operates in both a single CPU and/or multiple 
CPU environment with shared DASD. 

With TDMF, the implementation of new storage sub- 
system technologies can be accomplished in a nondisruptive 
manner. 

With TDMF, the value of existing storage equipment is 
protected because all devices can participate in a migration 
session. 

With TDMF, a competitive multiple vendor environment 
is enhanced because TDMF is designed to be independent of 
any specific storage subsystem. 
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1 

2 MODULE. TDMFMAIN 

3 The purpose of this module is to start the TDMF session on a Master or Slave system, open all necessary files, read 

4 and validate control cards, validate all parameters and volumes to be involved in the migration session. Initialize all 

5 TDMF control blocks and obtain the necessary storage for these control blocks. In the case of a TDMF Master system, 

6 the Communications Data Set (COMMDS) will be initialized to it's proper format and all parameters concerning the 

7 TDMF session recorded in the COMMDS. In the case of a Slave system, the parameter information concerning the 

8 TDMF session will be read from the COMMDS. This module executes in Supervisor State, Protect Key Zero and is 

9 the First part of the main task for an address space. 

1 0 This module contains a MVS Extended Set Task Abend Exit (ESTAE) recovery routine. This routine ensures that the 

1 1 TDMFMAIN task receives control in the event that the TDMF main task attempts an abend condition. 

n 

1 3 CSECT: TDMFMAIN 

1 4 Set up TDMF areas to include save areas, termination routines, message table, patch table, work areas and 

15 clock. 

16 Set up recovery environment to include input and output areas, file processing, and initiation/tennination. 

1 7 Obtain the SMFid of the executing system. 

1 8 Messages regarding volume processing are queued via routine ADD_MESSAGE for inclusion upon return 

19 to TDMFMAIN. Messages regarding s> stem processing are queued via routine ADD SYSTEM MESSAGE 

20 for inclusion upon return to TDMFMAIN. 

21 Call TDMFINTT. 

22 Start basic system. Determine if Master or Slave system. Obtain sufficient memory based on type of system. 

23 Call TDMFMSTR. 

24 Check the SMFID of the executing system, if a Master system. 

25 CallTDMFCOMl. 

26 Call TDMFCOM2. 

27 Call TDMFSLAV. 

28 Call TDMFVUCB. 

29 Call TDMFINSE. 

30 Call TDMFADDT. 

3 1 Initialize message area. 

32 If this is a Master system, Enqueue Exclusive, with a SCOPE=SYSTEMS, for all volumes in this migration 

33 session and pin the UCBs. This is done to prevent, in a multiple TDMF session environment, the attempt 

34 by a customer to migrate volumes involved in one TDMF session at the same time as another TDMF session 

35 is attempting to use the same volumes. If the customers Gl obal Resource Serialization { GRS) mechani s m does 

36 not provide support for multiple systems, the customer is required to execute all Master TDMF systems upon 

37 the same CPU complex. If this is the Master system, issue a call to module TDMFICOM requesting 
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1 initialization of the COMMDS . If this is a Slave system, issue a call to module TDMFICOM to update system 

2 record zero. 

3 Call module TOMFICOM requesting that all TDMF main system records to be written for this system. This 

4 includes the main TDMF control block, the TDMFTSOM control block, and information related to the TDMF 

5 authorization routine. This is done to provide diagnostic information that may be necessary at a later time. 

6 Call module TDMFVOL. 

7 This module will call module TDMFVOL, CSECT TDMFVOL, to complete all volume migration phases. 

8 Cancel the current ESTAEX recovery environment 

9 Go to CSECT TDMFTERM. 
10 ADD_SYSTEM_MESSAGE: 

1 i Add designated system message to queue of TDMF system messages located at the end of the main TDMF 

12 control block (TDMF), 

13 AI>I>_MESSAGE: 

1 4 Add designated volume message to queue of volume messages located in the TDMF volume message control 

15 block (TDMFVMSG). 

16 

17 CSECT: TDMFINIT 

18 If this is a Master system, set up the TDMF TSO Monitor (TDMFTSOM) control block and its embedded 

19 TDMF Monitor message (TDMFMMSG) area. Dynamically load the TDMF authorization module 

20 (TDMFAUTH) in to extended private. 

2 1 Determine and set level of Operating System. Mark TDMF as non-swappable, non-dispatchable, privileged, 

22 and exempt from load balancing within the operating system 

23 

24 CSECT: TDMFMSTR 

25 Check input records to determine validity, number of systems, number of volumes to migrate, and any 

26 customer selectable options* Save DCB from here for later use in TDMFCOM1 in order to limit the number 

27 ofDCBsused. 

28 

29 CSECT: TDMFCOM1 

30 Reuse DCB from TDMFINIT S YSIN processing to process COMMDS by resetting the pointer to SYSCOM. 

3 1 Verity that SYSCOM data set name and volume serial number is correct. Validate Format 1 DSCB and 

32 CCHHR in order to verify that COMMDS has contiguous cylinders allocated and that the allocation is large 

33 enough. 

34 Obtain UCB via the volume serial number and PIN the device in order to avoid deletion of the UCB via 

35 Dynamic Reconfiguration Manager (DRM). 

36 
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1 CSECT: TDMFCOM2 

2 Verify that COMMDS is not allocated on volume that is involved in a migration (Master only). 

3 initialize SRB/IOSB, CCW/IDAL, and COMM data areas. Obtain storage for COMM I/O and page fix it. 

4 Load I/O Monitor (TDMFIMON) into fixed CSA 

5 Load I/O code (TDMFSIO) into extended private. 

6 Load volume code (TDMFVOL) into extended private. 

7 Load volume I/O code (TDMFIVOL) into extended private. 

8 Load communications I/O code (TDMFICOM) into extended private. 

9 Load copy code (TDMFCOPY) into extended private. 
10 

11 CSECT: TDMFSLAV 

12 If this is a Slave system, obtain system record zero from COMMDS via a call to module TDMFICOM 

1 3 requesting a read of system record zero information. This provides all parameter information for the TDMF 

1 4 session for use by the Slave system, that has previously been initialized by the Master system. 

15 if this is a Slave system, verify Slave SMFid in system record zero in order to determine relative system 

1 6 number Compare Slave SMF SMCA to SMFid in system record zero. Determine if COMMDS has been used 

1 7 previously by this Slave system. 

1 g Determine if Stave has been submitted on same system as Master 

1 9 Initialize system message area (TDMFSMSG) and save SMFid and relative system number for Slave{s). 

20 Calculate number of systems involved with migration session and volumes that were identified by Master. 

2 1 Verify that number of volumes does not exceed maximum allowed. 

22 Initialize volume serial numbers in TDMF control block for UCB_LOOKUP processing, 

23 Issue exclusive ENQueue on COMMDS A'olser to ensure data integrity. 
24 

25 CSECT: TDMFVUCB 

26 Validate all source and target volumes in migration session. Verify that all UCBs exist for source and target 

27 volumes. Verify that UCBs do NOT exist for new volume serial numbers. Save UCB address information 

28 for PIN processing. 

29 Verify that source devices exist via the UCB, that they are online, not under-going a change in status (offline), 

30 and no page data sets exist on the volume<s). 

3 1 Verify that target devices exist via the UCB, that they are online, not under-going a change in status (offline), 

32 and nothing is allocated to the volume(s). The system will allow target devices to be non-existent (or offline) 

33 on a Slave system only, if the type of volume migration being requested is for a Point-In-Time migration. 

34 Save all migration session user specified options 
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1 If this is a Slave system, set up the TDMF TSO Monitor (TDMFTSOM) control block and its embedded 

2 TDMF Monitor message (TDMFMMSG) area. 

3 In order to allow the design of the quiesce mechanism of volume I/O operations that may be issued while the 

4 volume is quiesced, the TDMF design requires that the IOSLEVEL used by the operating system for Dynamic 

5 Device Reconfiguration (DDR) be saved for subsequent use by TDMFIVOL. 

6 Obtain storage subsystem information of source and target volumes so that re-chaining and update of the 

7 control blocks for those devices that have been swapped can be performed. Areas saved are SSSCB, 

8 SSCBDH, SSCBDV and SSCB device entries. 

9 Verify that COMMDS is allocated behind a control unit that supports ECKD CCWs. Determine device type 

10 an< i that the COMMDS is on the SAME physical device from the Master and Slave perspectives The 

1 1 information necessary to provide the above checks is retrieved by issuing various calls to module TDMFIVOL, 

12 requesting miscellaneous I/O operations to be performed against the volume that contains the COMMDS. 

13 

14 CSECT: TDMFENSE 

1 5 Initialize system entries for Master and Slave(s) (if present), TOD value, TDMF ASID, and "dead" intervals. 

16 Set system entries in COMMDS so that initialization will complete or disallow re-use of the COMMDS 

17 without Master re-initialization. 

1 8 Allocate a special set of TDMFVRNQ control blocks to be used as an input area, when the Master system 

19 reads, as a part of its normal processing in module TDMFVOL, CSECT TDMFCVOL, all TDMFVRNQ 

20 control blocks from all Slave systems 

21 

22 CSECT: TDMFADDT 

23 Obtain storage for TDMFWORK and its embedded TDMFDDTE control blocks. 

24 verify that the volume serial number matches the information in the UCB, via calling module TDMFIVOL, 

25 requesting the miscellaneous read volume label operation. Additional I/O operations are issued to collect 

26 information from the volumes involved in the migration session via subsequent calls to module TDMFIVOL. 

27 Among the I/O operations requested, are the miscellaneous requests to Read Device Characteristics, Sense 
2g Sense Subsystem Status. Then the volumes are checked to ensure that the volumes are not involved in 

29 a duplexing operation, volumes support ECKD CCWs, device type is supported, track geometries are equal, 

30 target device size is equal to or greater than the source device. 

3 1 Initialize various fields (TDMFMSV, TDMFMSVE, TDMFDDTE) and initialize request from Master to 

32 Slave(s). 

33 Obtain and initialize work areas (SRB/IOSB, CCW/IDAU Copy Cylinder I/O area, Copy Track I/O area, 

34 Event Notification Facility (ENF) work area; TDMFVMSG, TDMFVRNQ and TDMFVRBM control blocks), 

35 volume options (swaplpoint-in-time, autornatiefcrompt, single group, purge, and compare), andDDTE entries 

36 for source and target volumes. 

37 Determine D ASD subsystem capabilities (caching, Cache Fast Write (CFW), D ASD Fast Write (DFW)) and 

38 if they are active or inactive on the subsystem and/or volume. 

39 
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1 CSECT: TDMFTERM 

2 Set up recovery environment via an ESTAEX macro as previously documented In the case of an abnormal 

3 termination, determine who requested the termination {Master/Slave), what module detected the error 

4 (TDMFTMON, TDMFSIO, etc.), and if recovery via the TDMF main task is possible or to be passed to MVS 

5 Detach all copy tasks if Master system. 

6 Before termination of this system, if a reserve is still held upon the COMMDS, call module TDMFICOM, 

7 requesting a write/update of system record zero information, followed by a release of the COMMDS. 

8 Call TDMFICOM requesting the write of the TMFVMSG control block for all volumes from this system. If 

9 this is a Master system. Dequeue Exclusive for all volumes in this migration session and unpin the UCBs. 

1 0 Collect and print all TDMF TSO Monitor messages (TDMFMMSG), system messages (TDMFSMSG), and 

1 1 all migration messages {TDMFVMSG) for all volumes. If this is a Master system, collect and print all TDMF 

12 TSO Monitor messages (TDMFMMSG), system messages (TDMFSMSG). and all migration messages 

1 3 (TDMFVMSG) for all volumes that were issued on behalf of any Slave system that have been previously 

14 written to the COMMDS. This Slave system information is collected via calls to TDMFICOM collecting the 

15 appropriate control blocks from the COMMDS. 

16 Release and tree all storage areas obtained. 

17 Remove all TDMF modules from memory (TDMFIMON, TDMFCOFY, TDMFVOL, TDMFICOM, 
1 g TDMFI VOL, TDMFSIO, TDMF AUTH). Dequeue Exclusive Enqueue for the COMMDS volume and unpin 

19 the UCB. 

20 Cancel ESTAEX recovery environment and return control to MVS. 
21 

22 CSECT: TDMFMSG 

23 A repository' of all messages for TDMF as well as un-assigned messages. 
24 

25 CSECT; TDMFPTCH 

26 A patch area used for fixes. 
27 
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1 Module: TDMFVOL 

2 The purpose of this module is to control the migration process of any and all volumes being migrated within 

3 the TDMF session. It is called from module TDMFMAIN, CSECT TDMFMAIN, after the initialization of 

4 the COMMDS on a Master system or upon completion of all error checking on a Slave system. This module 

5 executes as a part of the main task for an address space and executes in Supervisor State, Protect Key Zero 

6 This module contains aMVS Extended Set Task Abend Exit (ESTAE) recovery routine. This routine ensures 

7 that the TDMFVOL task receives control in the event that the TDMFVOL task attempts an abend condition. 

8 

9 CSECT; TDMFVOL 

1 0 Create maximum wait of 15 minutes for initialization of slave system(s). , 

1 1 Call MS_ ADDRESSABILITY. 

1 2 Messages regarding volume processing are queued via routine ADD_MES S AGE for inclusion upon return 

13 to TDMFMAIN. Messages regarding system processing are queued via routine ADD_SYSTEM_MESSAGE 

14 for inclusion upon return to TDMFMAIN. 

15 BEGIN: 

16 CaJlTDMFCVOL. 

17 VOLUMEJPROCESSINGJLOOP: 

1 g Update statistical information for use by the TDMF Monitor. Volume processing handles the "phases" a 

19 migration goes through during it's processing life cycle. All routines return to 

20 NEXT_VOLUME_PROCESSING or to the appropriate phase. 

2 1 Logically, phase processing is executed in the following sequence: 

22 TDMFNVOL, TDMFAVOL, TDMFQVOL, TDMFS VOL, TDMFBVOL (ifTDMFS VOL fails), TDMFRVOL 

23 and TDMFTVOL. Resume processing (TDMFRVOL) must always be executed if the current volume 

24 migration status indicates that the volume is quiesced and thus, prior to execution of TDMFTVOL. 

25 There are sub-phases that may be executed asynchronously by an individual sub-task, one for each volume 

26 bei ng migrated, during the processing listed below. Each sub-phase executing as part of the individual sub- 

27 task communicates with the volume processing main task via posting of a sub-phase event control block 

28 (ECB) This sub-task provides three sub-phases within the domain of it's execution. These are the copy sub- 

29 phase, refresh sub-phase and synchronization sub-phase. 

30 After volume activation occurs, a copy sub-phase is used to copy all data from the source volume to the target 

3 1 volume A refresh sub-phase is executed at least once copying all data that may have been updated by a 

32 Master or Slave system for that volume. This design allows customer access to all data on the source volume, 

33 including update activity, during the life of the migration process. The TDMF I/O Monitor module 

34 (TDMFIMON) dynamically detects that a Master or Slave system has updated data located on the source 

35 volume and communicates this information to the TDMFVOL main task to be used by CSECT TDMFMVOL. 
36 

37 Prior to volume swap processing, additional refresh sub-phases maybe necessary to provide a iterative process 

38 since data upon a migrating volume could again have changed by activity on any system during execution of 

39 the previous sub-phase Special processing can cause the execution of the quiesce phase, resume phase, and 
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1 refresh sub-phase processing until TDMF dynamically determines that the synchronization sub-phase can 

2 completely synchronize the migrating volume{s) based upon time goals specified by volume migration 

3 parameters. 

4 In actuality, phases are called and checked in reverse hierarchical sequence from the logical phase processing 

5 previously documented because if the phase does not need to be processed, for specified volume, the next 

6 lower phase is executed via a GO TO statement for the next logical phase: 

7 • TDMFTVOL - termination processing 

8 • TDMFRVOL - resume processing 

9 • TDMFBVOL - back-out processing 

10 • TDMFS VOL - swap processing 

11 • TDMFQVOL - quiesce processing 

12 • TDMFAVOL - activate processing 

13 • TDMFNVOL - initialize processing 

1 4 Each volume control block (TDMFMSV) contains a word indicating the phases that have been completed or 

15 are being requested to be executed upon each of the possible systems. Each phase is represented by a word 

16 (32 bits), called the MSV_(phase)MASK. The MSV„(phase)MASK field is initialized to a value of one in 

1 7 all system defined bit positions that do not exist in the TDMF session. Processing for an individual phase and 

18 it's corresponding mask, is handled by the appropriate CSECT. For example, swap processing is handled in 

19 TDMFSVOL and is controlled with bits in MSV_SMASK. 

20 The appropriate MSVJphase)MASK bit zero defines that the Master system is requesting the function. A 

2 1 Slave system does not attempt execution of the phase if the Master system has not requested the function {bit 

22 zero of the MSV_(phase)MASK is a zero). Bits 1 through 31 are set when the respective slave system 

23 completes the phase that has been requested by the Master. The Master system normally only executes the 

24 requested phases when all slave systems have responded that the phase has completed. Thus, all bits are set 

25 to one. Upon completion of the phase on the Master system, an indicator is set in a volume flag 

26 (MSV_VOL_FLAG) indicating that that phase is complete. Processing will continue to the next phase 

27 Effectively, this TDMF design provides a positive acknowledgment environment, i.e., that all Slave systems 

28 only execute the functions requested by the Master system. The Master system only executes the requested 

29 phase upon successful completion by all Slave systems for that phase. On the other hand, there are two cases 

30 in which a Slave system might like to alter the phase processing sequence, but since a Slave system cannot 

3 1 request processing of a specific phase as bit zero can only be set by the Master system, the Slave system must 

32 have a way to notify the Master to alter it's phase processing sequence. 

33 The first case is when a Slave system detects an error via the TDMF I/O Monitor which requires volume 

34 termination processing of the migration be invoked. 

35 The second case is that a Slave system detects an error during swap processing which requires that back-out 

36 processing be invoked. Since other Slave systems within the TDMF migration session may be present, and 

37 may have successfully executed swap processing for the volume, a swap volume processing failure on one 

38 Slave system must be communicated via the Master system, to all Slave systems, to invoke back-out 

39 processing. 
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1 Thus, in these two cases, a Slave system may turn on its designated system bit in MSVJTMASK or 

2 MSV_BMASK requesting the Master system to effectively request these functions on all systems. Upon 

3 recognition on a Master system the Slave system designated bit is reset to zero by the Master system after the 

4 requested functions bit zero is set This allows the Slave systems designated system bit to be again used as 

5 an acknowledgment indicator indicating that the Slave system has completed the requested function. 

6 During volume phase processing, the Master system examines MSVJTMASK and/or MSVBMASK at the 

7 appropriate points t0 determine if a Slave system has set their respective bit in MSV_TMASK or 

8 MSV_BMASFC indicating to the Master to alter it's volume processing sequence 

9 Branch to TDMFTVOL. 

1 0 NEXT_VOLUME_PROCESSING: 

11 if this volume has completed all phases, go to NEXT_ VOLUME. If volume termination is being requested, 

12 go to NEXT_ VOLUME. If TDMF is executing on a Slave system, go to NEXT_ VOLUME. If the copy sub- 

13 task does not exist, go to NEXT_ VOLUME. If the copy sub-phase has been posted, go to 

14 CHECKCOPYJSNDED. 

15 NEXTVOLUME: 

1 6 Update statistical information for use in the TDMF Monitor. If any system appears to have failed during the 

17 life of the migration, go to NEXT_ VOLUME_PROCE S SIN G_S Y S TEM_DE AD in CSECT TDMFCVOL. 
18 

1 9 if the system is still active, increment all system control block pointers to their next respective entries and if 

20 all volumes have not been processed, go to VOLUME_PROCESSING_LOOP. If all volumes have been 

2 1 processed, determine the amount of time that this system should wait before re-awakening. 

22 Go to TDMFUVQL. 

23 COMMON ROUTINES THAT ARE USED By ANY OF THE ABOVE PHASES: 

24 SETJRESUME: 

25 If a resume I/O to the source volume is being requested, set the request function to resume. Test for number 

26 of systems, if one, go to TDMFRVOL, else go to NEXT_VOLUME_PROCESSTNG 

27 CHECK_SYNCHRONIZATION_COMPLETE: 

28 Set flag based on the copy task (compare/synchronize) function when the copy synchronization process is 

29 complete. If copy task does not terminate normally, check to see if volume (source) I/O is quiesced. If so, 

30 resume I/O to source volume and/or terminate processing. 

31 
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1 SETJSWAP: 

2 If a swap of the source volume is being requested, set request function to swap. Test for number of systems. 

3 If one, go to TDMFS VOL, else continue to NEXT_VOLUME_PROCESSING. 

4 If the volume is not currently quiesced, go to NEXT_ VOLUME. Else, set request function to quiesce once 

5 the refresh end function has completed. If slave systems are involved, the Master will wait until notification 

6 from Slave<5) that quiesce request has been processed, before continuing. If automatic migration, quiesce and 

7 synchronize without intervention. If prompt or single group option was selected, create addressabil ity to the 

8 TDMF Monitor control block (TDMFTSOM), pass indicator to TDMF Monitor that a prompt is required. 

9 Wait for response, then quiesce and begin the synchronization sub-phase. 

10 CHECK_COPY_ENDED: 

U If the copy sub-phase ended normally, go to NEXT_VOLUME. Since the copy sub-phase did not end 

1 2 normally, ensure that the volume is not quiesced. If quiesced and resumed, set volume termination request 

13 function. Else, go to NEXT VOLUME. 

14 SETTERMINATION: 

15 if a termination of the source volume is being requested, set request function to terminate. Ensure thai all 

1 6 Siave systems MS V_TMASK respective bit is set to zero so that all Slave systems will not believe that they 

1 7 have completed slave processing in case one of the slave systems was requesting the Master system to alter 

1 8 it's normal volume processing sequence. Terminate the copy sub-task for this volume's migration. Test for 

19 number of systems, if one, go to TDMFTVOL, else go to NEXT_VOLUME PROCES SING. 

20 ABEND_TERMINATION JPROCESSIN G: 

21 Set abend termination as a severe error, indicating if the error is recoverable or unrecoverable, and the type 

22 of system, Master or Slave, upon which error occurred. 

23 If a read/reserve is outstanding, this means record zero is still in memory. Go back and process volumes again 

24 in order that normal update of system record zero and release of the COMMDS, processing will complete. 

25 If not, re-read record zero. 

26 UPDATE STAT1STICAL FNFORMATION: 

27 Set the time interval between updates to the Communications Data Set (COMMDS) based on phase: 

28 • 30 seconds - copy phase (default) 

29 • 15 seconds - 1 st refresh pass 

30 • 10 seconds - 2 nd refresh pass 

31 • 5 seconds - 3 ri and subsequent refresh passes 

32 • 1 second- quiesce, synchronize, swap and resume processing 
33 

34 A TDMF systems response time is set to be normally the minimum time interval from the above, based upon 

35 every volumes phase. 

36 TDMF was designed to be as responsive as possible without placing a large burden upon the CPU, On the 

37 other hand, there are times that the TDMF system must be extremely responsive to minimize the impact to 
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; 1 customer application I/O operations that have been quiesced and lasting until the I/O operations have been 

2 resumed. This includes the time required to do volume quiesce phase processing, volume synchronize sub- 

3 phase processing, volume swap phase processing, and volume resume phase processing. This is why the 

4 systems response time is set to one second if any volumes migration is quiesced and not resumed, which 

5 encompasses the aforementioned phases and sub-phase. 

6 Therefore, TDMF will only be as responsive as required based upon necessity. 

7 Calculate the elapsed time of processing that has been used by the TDMFVOL main task during this 

8 execution. Calculate the elapsed time for each phase as well as total clock time for migration. Calculate the 

9 average device response time for each volume migration. These values are used in the Monitor. 

10 CALLTDMFICOM: 

1 1 Call the TDMF I/O communications routine (module TDMFICOM) to execute the requested read or write 

1 2 operation. If TDMF is executing on a system that has been previously determined to have been "dead", the 

13 I/O operation is allowed only if it is a wri te I/O operation and the write I/O operation is not attempting to re- 

1 4 write system record zero which contains all TDMFMSE, TDMFMSV, and TDMFMSVE information. This 

1 5 will prevent a system that has "died" from re-awakening and destroying system record zero information that 

1 6 may be in use by the recovery process and it's subsequent termination of the TDMF system. 

17 CALL TDMFTVOL: 

1 8 Call the TDMF volume I/O routine (module TDMFTVOL) to execute the function being requested. 

19 CALL_UCB_VOL: 

20 The purpose of this routine is to pin or unpin a MVS Unit Control Block (UCB) as requested by the caller. 

2 1 The purpose of the pin or unpin is to prevent MVS from invoking Dynamic Reconfiguration Management 

22 (DRM) functions, thereby changing the address of the UCB during the volume migration process 

23 MOIHFYJJCBDDT: 

24 Determine each source and target volumes MVS UCBs Device Dependent Table (DDT) via the common 

25 extension in the UCB. Processing will depend on a caller option of chain or unchain. 

26 An integral design consideration in TDMF requires that TDMF dynamically modify the Device Dependent 

27 Table (DDT) used by MVS' IOS component such that the TDMF I/O Monitor (module TDMFIMON) receives 

28 control during customer I/O operations against the migrating volumes during the life of the volume 

29 migrations In order to provide thisfunction transparently, a TDMF control block (TDMFDDTE) is allocated 

30 for each source and target volume embedded within the TDMFWORK control block. Current standard MVS 

3 1 information located in MVS* DDT is copied to the newly allocated TDMFDDTE and additional information 

32 is placed into the TDMFDDTE to ensure that the TDMF I/O Monitor (module TDMFIMON) receives control 

33 at the proper time for it's processing. The chaining or unchaining operation being requested dynamically 

34 modifies the standard MVS UCB to point to the TDMFDDTE during a chain operation or reset to point to 

35 the standard MVS DDT for an unchain operation. 

36 The address of MVS' standard Start I/O (SIO) Device Dependent Exit in MVS ' standard DDT is changed to 

37 point to the TDMFIMON main entry point, and it's address is saved in the TDMFDDTE control block. This 

38 allows the TDMFIMON routine to receive control at the proper time during MVS ' Input/Output Supervisor 

39 (IOS) processing transparently. 
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1 If there was an error detected in TDMFIMON, the Functional Recovery Routine (FRR) in TDMFIMON will 

2 have already reset the MVS DDT pointer in the UCB and indicated to TDMFVOL and TDMFCOPY sub-task 

3 processing that they should terminate that volumes migration with errors. 

4 During de-activation of TDMFIMON, it is possible that an I/O operation is still ninning and has not had it's 

5 I/O interrupt processed. To be sure, the last Input/Output Supervisor Block (IOSB) is checked for the 

6 Input/Output Supervisor Disable Interrupt Exit (IOSDIE) routine pointer. If the IOSDIE routine pointer still 

7 points to the DIE entry point at TDMFIMON-J-4, it is replaced with the normal MVS IOSDIE routine address 

8 which will be saved by TDMFIMON during its processing as a part of device dependent SIO routine. 

9 CLEANJOSLEVEL: 

10 A TDMF design consideration requires that TDMF be able to quiesce and prevent customer I/O operations 

1 1 from being executed to the source and target volumes during critical phases of a volumes' migration. TDMF 

1 2 implements this requirement based upon the feet that MVS IOS will not allow an I/O operation, represented 

13 by an IOSB, who's IOSLEVEL value is numerically less than the UCBLEVEL field of the UCB to which the 

14 I/O operation is being directed. 

1 5 Normal I/O operations have an IOSLEVEL of X'01 '; the UCB has a UCBLEVEL of X'01 \ During quiesce 

16 processing, the UCBLEVEL will be changed via an IOSLEVEL macro to the Dynamic Device 

1 7 Reconfiguration (DDR) level of the MVS operating system. The UCBLEVEL is effectively a priority which 

1 8 only allows subsequent I/O operations, represented by an IOSB, of equal or higher priority tobe issued against 

19 the migrating volume. 

20 The purpose of this routine is to ensure during volume termination processing that a UCB is not inadvertently 

2 1 left with a UCBLEVEL which is not equal to the normal UCBLEVEL of X'O V. 

22 DEACTIOMON: 

23 The TDMF I/O Monitor (module TDMFIMON) examines the TDMFDDTE to determine if the TDMF system 

24 still requires the module to monitor I/O operations being executed against the migration volumes. This 

25 routine indicates to TDMFIMON that monitoring of I/O operations is no longer required. 

26 MS_AD DRESS ABILITY: 

27 Set up standard system addressing to the system control block for the system (Master or Stave) upon which 

28 TDMF is being executed (TDMFMSE), the first volume control block (TDMFMSV) representing the first 

29 migration, and the first volume entry control block (TDMFMSVE) for the first volume on the this system. 

30 Next, set up addressing to a work control block (TDMFWORK) which contains addressing capability to the 

3 1 Device Dependent Table Extended (TDMFDDTE). 

32 
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1 TS_ ADDRESS ABILITY: 

2 Provide addressability to the control block to communicate to the TDMF TSO Monitor. This control block 

3 provides a communications vehicle between the TDMF main task and the TSO user executing the TDMF 

4 Monitor. 

5 RESERVE_COM: 

6 In order to allow multiple systems in a TDMF session to communicate with each other, the status of all 

7 volumes and all systems during volume migration processing, is recorded in the COMMDS. To protect the 

8 data integrity of the data within the COMMDS, the TDMF system logically provides protection via a standard 

9 MVS reserve or enqueue macro and physically provides protection with an actual hardware reserve channel 

10 command word which will be issued during the read of system record zero. This will prevent multiple TDMF 

1 1 systems from attempting to use the information recorded in the COMMDS at the same instant in time. 

12 RELEASE_COM; 

13 At the end of the update of system record zero in the COMMDS, a logical MVS dequeue operation is 

14 performed to indicate that the COMMDS may be accessed by another TDMF system. On the other hand, the 

1 5 actual physical release operation is dependent upon whether the MVS operating system has a physical reserve 

1 6 upon the volume belonging to another application using the same DASD volume containing the COMMDS 

1 7 MVS will not release the physical protection until no applications still require any physical volumeprotection. 

1 8 ADDSYSTEMJV1ESSAGE: 

19 Add designated system message to queue of TDMF system messages located at the end of the main TDMF 

20 control block (TDMF). 

21 ADDJWESSAGE: 

22 Add designated vol time message to queue of volume messages located in the TDMF volume message control 

23 block (TDMFVMSG). 

24 LOCKLOCAL: 

25 Obtain the MVS local lock. 

26 LOCK.CMS: 

27 Obtain the Cross Memory Services (CMS) lock. 

28 UNLOCKLOCAL: 

29 Release the MVS local lock. 

30 UNLOCK_CMS: 

31 Release the CMS lock. 

32 UCB_UNALLOCATE: 

33 Allow the TDMF system to clear allocation information placed in the UCB during activate volume processing 

34 in CSECT TDMF A VOL. 

35 
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1 TDMF.RECOVERY: 

2 This is the entry point for the MVS Extended Set Task Abend Exit (ESTAE) recovery routine. This routine 

3 ensures that the TDMFVOL module receives control in the event that the TDMF mai n task attempts an abend 

4 condition. 

5 

6 CSECT; TDMFCVOL 

7 Set up parameters to read the COMMDS, system record zero consist! ng of all information in the TDMFMSE, 

8 TDMFMSV and TDMFMSVE control blocks. In addition, if this is the Master system, request that the I/O 

9 operaUon also read all Volume Refresh Notification Queue (TDMFVRNQ) control blocks. With one I/O 

1 0 operation, this system record zero i nformation and all TDMFVRNQ control blocks will be read by calling the 

1 1 RESERVE_COM and CALLJCOM routines. 

12 READ COMPLETED OK: 

1 3 Update the system entry heartbeat to indicate that this TDMF system is still "alive". 

1 4 There is a 1 5 minute time interval before another system may believe that this system is "dead". If another 

15 system believes that this system is "dead", the VOLUME_PROCESSlNG_SYSTEM_DEAD routine, located 

16 in this CSECT, will be invoked on all systems that detect that this system is "dead". These routines will 

17 terminate all volume migrations, in a controlled fashion, based upon each volumes" current migration phase 

18 or sub-phase, for each system, in the TDMF life cycle sequence. These routines will modify the normal 

1 9 volume processing loop described in CSECT TDMFVOL previously. 

20 Determine if another system has previously detected that this TDMF system was ever considered "'dead". The 

2 1 only way that a system may appear "dead" to another system, is if the logical processor was stopped physically 

22 and while this system did not hold a reserve on the COMMDS. If this is true, it may be possible to start 

23 TDMF processing by having the logical processor re-started. Therefore, this TDMF system, which has been 

24 considered "dead" and has re-awakened, will invoke a specialized volume processing loop to terminate all 

25 volume migrations consistent with the recovery processes used by all systems who had considered this system 

26 '"dead". Those processes are described in the VOLUME_PROCESSING_SYSTEM_DEAD, located in this 

27 CSECT and are dependent upon whether the system that "died" (this system), is a Master or Slave. 

28 If this system was not previously considered Mead" and re-awakened, then determine if this is a Master or 

29 Slave system. 

30 If this is a Slave system, fetermine if the Master system is still "alive". If the Master is still "alive", return 

3 1 to VOLUME_PROCESSING_LOOP in CSECT TDMFVOL. 

32 If this is a Master system, determine if any Slave system is "dead". If all Slave systems are still "alive", return 

33 to VOLUME J>ROCESSING„LOOP in CSECT TDMFVOL. 

34 VOLUJViE_PROCESSING_SYSTEM_DEAD: 

35 If any system detects that a system is "dead", mark the appropriate systems entry that it is considered "dead", 

36 and set theMS VJphase)MASK to the appropriate phase to cither terminate the volume migrations, back-out 

37 the volume migrations, or to consider the volume migrations complete if swap processing has completed. 

38 Ensure that any volume that has had I/O operations quiesced to it, be resumed. 

39 NEXT_VOLUME_PROCESSING_SYSTEM_DEA»: 
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1 Increment all system control blocks to their next respective entries and if all volumes have not been processed, 

2 go to VOLUME JPROCES SING_S YSTEM_DE AD . If all volumes have been processed, detennine the 

3 amount of time that this system should wait before re-awakening. 

4 Go to TDMFUVOL. 
5 

6 CSECT; TDMFTVOL 

7 The purpose of this phase is to terminate the volume processing for a specific volume. 

8 If volume processing is complete for this volume, go to NEXT_ VOLUMEPROCES SING in CSECT 

9 TDMF VOL. 

1 0 Determine if the TDMF I/O Monitor (module TDMFIMON) is still active and monitoring for I/O operations 

11 for this specific volume via a call to CHECK EVi ON . If the TDMF I/O Monitor is still active, go to 

1 2 MONITOR_ ALIVE. If any error occurs during TDMFIMON processing, TDMFIMON will signal the error 

1 3 condition by turning off a bit in the TDMFDDTE control block indicating that the I/O Monitor is no longer 

1 4 active. Therefore, if the I/O Monitor is no longer active, go to MONITORDEAD. 

15 MONITOR_ALIVlS: 

1 6 If the TDMFVOL task has not abended on this system, go to THIS_SYSTEM NOT LENDING. 

17 MONITORDEAD: 

1 8 If this is a Slave system, request termination of the specific volume migration by setting this systems bit in 

1 9 the MSVTMASK field. This effectively asks the Master to terminate this volume's migration. 

20 If this is a Master system, and the volume is quiesced and not resumed, go to SET_RESUME in CSECT 

2 1 TDMFVOL. If the volume is not quiesced or was quiesced and then later resumed, set the request termination 

22 function for this volume and terminate the copy task. 

23 THIS_SYSTEM_NOT_ABENDING: 

24 If this is a Master system, go to CHECKFORSL A VE TERMIN ATION REQUE ST. Otherwise, this is a 

25 Slave system which will examine the Master systems entry to detennine if a failure occurred upon the Master 

26 system and that the Master system requested system termination of the TDMF session on all Slave systems. 

27 If volume termination processing has completed for this Slave system, go to 

28 NEXT_VOLUME_PROCESSING in CSECT TDMFVOL. If volume tennination processing has not 

29 completed upon this Slave system, go to VOLUME_TERMINATION_PROCESSING_SLAVE. 

30 CHECK_FOR_SLAVE_TERMINAT10N_REQUEST: 

31 If this is a Slave system, go to NORMAL_TERMINATlON_CHECK. Determine if any Slave system 

32 requested termination due to an initialization failure. Detenrune if any Slave system requested termination 

33 of the TDMF session. If either is requested, terminate the copy task and go to GIVE_SLAVE_TERM_MSG. 

34 TEST_SLAVE_VOLUME_TERMlNATION_REQUEST: 

35 If the volume termination request function has been set, go to NORMAL TERMINATION CHECK. If any 

36 Slave system has requested volume back-out processing for this volume, set that Slave systems back-out 

37 volume request bit in MSVJBMASK to zero Set the request function in MSVJBMASK to one to cause all 

38 Slave systems to invoke volume back-out processing. Go to NORMAL_TERMINATION_CHECK. 
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1 NOT BACKOUT_REQUEST: 

2 Determine if any Slave system has requested termination of this volume migration If no, go to 

3 NORMAL_TERMINATION_CHECK. If any Slave system has requested volume termination processingfor 

4 this volume, set that Slave systems volume termination request bit in MSVJTMASK to zero. Set the request 

5 function bitin MSVTMASK to one to cause all Slave systems to invoke volume termination processing. Go 

6 to NORMAL_TERMINAT10N_CHECK. 

7 G WE_SLA VE_TERM_MS G : 

8 If the volume termination request function has been set, go to NORMAL_TERMIN AT!ON_CHECK 

9 Terminate copy sub-task. If the synchronization sub-phase ECB is waiting to be posted, post the sub-phase 

10 telling the sub-phase to terminate, then go to TERM_QUTESCE_CHECK. If the refresh sub-phase ECB is 

1 1 waiting to be posted, post the sub-phase telling the sub-phase to terminate 

12 TERM_QUIESCE_CHECK: 

13 if me volume has been quiesced, go to CHE CK S YNCHRON1Z AT ION COMPLETE . Set the termination 

14 request function in MSVJTMASK. If the number of systems is not equal to one, go to 

15 NEXT_VOLUME_PROCESSTNG in CSECT TDMFVOL. 

16 NORMAL JTERMINATION.CHECK: 

17 if this is a Slave system, go to SKIP_CUSTOMER_TERMTNATE. Get addressability to the TDMFTSOM 

18 control block. If TDMF has already acknowledged a change in the synchronization goal time value, go to 

1 9 TE ST_CU STOMER_TERMlN ATE . If the customer has not requested a change in volume synchronization 

20 goal time, go to TEST_CU STOMER JTERMN ATE. Acknowledge volume synchronization goal time change 

21 and save the value for the synchronization goal. 

22 TEST_CUSTOMER_TERML\ATE: 

23 If TDMF has already acknowledged a volume termination request, go to S K3P_CTJ STOMER TERMINATE . 

24 If the customer has not requested volume termination, go to SOP_CUSTOMER_TERMINATE. If the 

25 customer has requested volume termination, acknowledge the volume termination request and go to 

26 GIVE_SLAVE_TERM_MSG. 

27 SKIF_CUSTOMERTERMIISATE: 

28 If volume termination is not requested (MS VTMASK, bit zero set), then go to TDMFRVOL. If any Slave 

29 system has not processed the volume termination request, go to VOLUME_TERMINATlON_SLAVE. If this 

30 is a Slave system, go to NEXT J/OLUME JTOCESSING. 

31 
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1 VOLUME TERM INATION_PROCESSING_MASTER: 

2 Call DEACTJOMON. If all Slave systems have not acknowledged volume termination processing, go to 

3 VOLUME_TERMINAnON_PROCESSING_SLA VE. Set the volume termination and complete flags in 

4 MSV_VOL_FLAG. Release page frames associated with copy cylinder I/O areas. Decrement the number 

5 of volume migrations active on this system. If there are still active volumes being migrated, go to 

6 CHECK_UCB_ ALLOCATION. If the number of systems that are active is zero, go to 

7 CHECK_UCB~ ALLOCATION. Decrement the number of systems active and mark this system as inactive. 

8 Go to CHECK~UCB_ALLOCATION. 

9 VOLUME_TERMINATION_PROCESSING_SLAVE: 

10 If this is a Master system, go to NEXTJVOLUMEJ>ROCESSING. If this Slave system has completed 

1 1 volume termination processing for this volume, go to NEXT_VOLUME_PROCESSING. 

1 2 VOLUIWE_TERMINATION_PROCESSING_MASTER_DEAD: 

1 3 Call DEACTJOMON. Set this systems volume termination complete in MSVTMASK. Decrement number 

14 of vol umes active on this system . 

1 5 CHECKUCB_ALLOCATION: 

16 Call CLE ANIO SLE VEL for the target volume. Call MODIFYUCBDDT for the target volume requesting 

17 an unchain operation. Call CLEANJOSLEVEL for the source volume. Call MODIFYUCBDDT for the 

18 source volume requesting an unchain operation. Call LOCK_LOCAL. Call LOCK CMS. Call 

19 UCBJJN ALLOCATE for the target volume. Call UCB UN ALLOCATE for the source volume. Call 

20 UNLOCK_CMS. Call UNLOCK_LOCAL. Go to NEXl'rVOLUME_PROCESSING. 

21 CHECKJMON: 

22 If the volume migration is not active, via a check of MSV AMASK on the Slave system, or 

23 MSV_VOL_FLAG on a Master system, return to the caller. If the volume is backed-out or swapped, return 

24 to the caller. If the I/O Monitor is in "quiet mode" and not currently monitoring for I/O operations, return 

25 to the caller If the I/O Monitor is not active, return to the caller signaling that the I/O Monitor is "dead". 

26 Invoke termination of volume migration. This check is made for both the source and target volumes in the 

27 migration pair. If the TDMFDDTE control block is not properly chained to the MVS UCB, return the same 

28 error condition. 

29 

30 CSECT: TDMFRVOL 

3 1 The purpose of this phase is to resume the volume processing for a specific volume. 

32 1b order minimize the impact to customer I/O operations, the amount of time that a volume cannot 

33 successftilly execute I/O operations (from the time that the volume is quiesced until the volume is resumed), 

34 is kept to a minimum. Normally, the Master system waits for a positive acknowledgment from ail Slave 

35 systems prior to executing a function that has been requested. The TDMF system design allows the resume 

36 operation to take place immediately upon the Master system without waiting for a positive acknowledgment 

37 from all Slave systems. This nunimizcs the impact above. On the other hand, the normal next phase is the 

38 volume termination phase, and that phase will not be allowed to be requested until the Master system has 

39 received a positive acknowledgment from all Slave systems that the volume has successfully resumed 
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1 If resume processing has not been requested via MSVJRMASK, go to TDMFB VOL. If this system has 

2 already resumed processing for this volume as indicated in MS V_RMASK or in MSV_VOL_FLAG, then go 

3 to NEXT_VOLUME_PROCESSING in CSECT TDMFVOL. 

4 If this is a Master system and this volume is not involved in multiple refresh phases, remove the TDMF I/O 

5 Monitor from "quiet mode". Resume I/O operations to customer applications via the issuance of an 

6 IOSLEVEL MVS macro to reset the UCBLEVEL from the DDR level to the normal level for both the source 

7 and target volumes. Indicate that the volumes are resumed via setting a resume indicator in 

8 MSV_VOL_FLAG or the respective system bit in MSVJRMASK. If this is a Slave system, go to 

9 NEXT_VOLUME_PROCE S SING in CSECT TDMFVOL. 

10 If this Master system has not received a positive acknowledgment from all Slave systems that this volume has 

1 1 successfully resumed, go to NEXT_VOLUME_PROCES SING in CSECT TDMFVOL. If another refresh sub- 

12 phase is requiredfor this volume, reset the MSVJRMASK and the MSV_QMASK indicating that this volume 

13 i s no longer quiesced, nor resumed. Decrement the volume time interval that system should awaken for 

14 processing of this volume by seconds without allowing the value to go below the value of 5 seconds. 

15 Increment the refresh number and post the synchronization sub-task ECB to start another refresh sub-phase. 

16 Go to NEXT_VOLUME„PROCESSING in CSECT TDMFVOL. 

17 

1 8 CSECT: TDMFBVOL 

19 The purpose of this phase is to back-out the volume processing for a specific volume. 

20 If back-out processing has not been requested via MSV_BMASK, go to TDMFSVOL. If this system has 

21 already performed back-out processing for this volume as indicated in MSV_BMASK or in 

22 MS V_VOL_FL AG, then go to NEXT JVOLUME_PROCESSING in CSECT TDMFVOL. 

23 Volume back-out processing is invoked due to one of two possible failures during swap processing. The first 

24 possible failure can be due to a failure while re-writing the volume serial numbers on the source and target 

25 volumes. The second possible failure can occur during the actual swap process when the addresses of the 

26 UCBs are modified. 

27 If this system does not require back-out processing from the swap function, go to 

28 RE WRITE_VOLUME_L ABELS . If the source volume is a JES3 system managed device, issue a Subsystem 

29 Function Request to XES3 to validate and indicate that a swapfiinction is starting and that the UCB addresses 

30 involved in the back-out processing are correct and that back-out processing may continue. Invoke back-out 

31 processing by executing procedures documented mCSECTTDMFS VOL, in order to reverse the original swap 

32 process. If the source volume is a JES3 system managed device, issue a Subsystem Function Request to JES3 

33 to notify JES3 that the swap processing is complete. 

34 Ensure that both the source and target volumes are ready for I/O operations as the previous swap process 

35 marks the devices as not ready. 

36 
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1 REWRJTE_VOLUME_LABELS: 

2 If this is a Slave system, mark back-out volume processing complete in MSV BMASK for this system, and 

3 go to NEXT_VOLUME_PROCESSING in CSECT TDMFVOL. Call CALL JTOMFI VOL passing a request 

4 of write volume label with the addresses of the source and target UCBs in order to re^write the volume serial 

5 numbers. Mark the target volume offline. Set appropriate bit in MSV_VOL_FLAG to indicate that back-out 

6 processing is complete for the volume. Set the resume request function bit in MS V_RMASK. If the number 

7 of systems is equal to one, go to TDMFRVOL. 

8 

9 CSECT: TDMFSVOL 

1 0 The purpose of this phase is to perform swap processing for a specific volume. 

11 If swap processing has not been requested via MS V_SMASK, go to TDMFQVOL. If this system has already 

1 2 completed swap processing for this volume as indicated in MSV_SMASK or in MSV VOL FLAG, then go 

1 3 to NEXT_VOLUME_PROCESSING in CSECT TDMFVOL. 

14 Call LOCKJLOCAL. Call LOCK_CMS. Call UCB UN ALLOCATE for the source and target volumes. Call 

15 UNLOOCjCMS Call UNLOCK~LOCAL. 

1 6 There are two types of migration that may be requested by the customer. If a Pouit-In-Time migration has 

17 been requested, go to SWAP_COMPLETE. If a Swap migration has been requested, continue. 

1 8 Initialize all return codes for Event Notification Facility (ENF) calls, swap processing, back-out processing, 

19 and JES3 Subsystem Request Functions. 

20 If the source volume is a JES3 system managed device, issue a Subsystem Function Request to JESS to 

21 validate and indicate that a swap function is starting and that the UCB addresses involved in the swap 

22 processing are correct and that swap processing may continue. 

23 Standard MVS processing invokes a module called IOSVS WAP. Information located in the UCB Device 

24 Class Extension (DCE) for the source and target volumes may not be correct in a standard MVS environment. 

25 The target volumes information may not be available as the device may be offline. The TDMF design requires 

26 that the source and target volumes be online during the life of the TDMF migration process. This design 

27 therefore, eliminates the possibility of the UCBs DCE containing incorrect information. 

28 TDMF is designed to ensure that the UCB DCE information is properly swapped between the source and 

29 target volumes. Thus, prior to the swap process, the UCB D CE information is saved for the source and target 

30 volumes. 

31 WRITEVOLUMELABELS: 

32 Call CALL_TDMFIVOL passing a request of write volume label with the addresses of the source and target 

33 UCBs in order to write the proper volume serial numbers. 

34 SWAP.PROCESS: 

35 Call LOCK_LOCAL. Call LOCK_CMS. Call MVS module IOSVS WAP passing the addresses of the source 

36 and target UCBs to perform swap processing. 

37 Call UNLOCK CMS. Call UNLOCK.LOCAL. If swap processing completed normally in module 

38 IOSVSWAP, go to FIX_UCB_NEXT. if swap processing failed in module IOSVSWAP, and the source 
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1 volume is a JES3 system managed device, issue a Subsystem Function Request to JES3 to notify JES3 that 

2 swap processing completed unsuccessfully. 

3 If this is the Master system, set the back-out request function in MSV_BMASK, and go to 

4 NEXT_VOLUMEJ*ROCESSING in CSECT TDMFVOL. 

5 If this is a Slave system, mark that this Slave system requests back-out volume processing in MSVBMASK 

6 for this system, and go to NEXT_VOLUME_PROCESSING in CSECT TDMFVOL. 

7 F1X_TJCB_NEXT: 

8 Ensure that both the source and target volumes are ready for I/O operations as the previous swap process 

9 marks the devices as not ready. 

10 Mark the original source UCB offline. Set the original source UCB volume serial number to that of the new 

1 1 volume serial number if the customer so specified. Use a compare and swap double loop to swap the UCB 

1 2 DCE information between the original source and target UCBs from the saved DCE information. If the source 

13 volume is a JES3 system managed device, issue a Subsystem Function Request to JES3 to notify JES3 that 

14 the swap processing is complete. Use standard ENF routine to notify MVS and any ENF "listeners" that a 

15 DDR swap operation has occurred involving the source and target volumes. 

16 If this is a Slave system, indicate that swap volume processing is complete in MSV_SMASK for this system, 

17 and go to NEXT_VOLUME_PROCESSING in CSECT TDMFVOL. 

18 Set appropriate bit in MS V_VOL_FL AG to indicate that swap processing is complete for the volume. Set the 

1 9 resume request function bit in MS V_RMASK. If the number of systems i s equal to one, go to TDMFRVOL. 

20 

21 CSECT: TDMFQVOL 

22 The purpose of this phase is to perform quiesce processing for a specific volume. 

23 If quiesce processing has not been requested via MSVQMASK, go to TDMF A VOL. 

24 TEST_DEVICE_STATUS: 

25 Call the IOSGEN_ROUTINE to determine if the quiesce function may continue for the source volume. If the 

26 return from IOSGEN_ROUTINE indicates that quiesce processing may not continue, retry 50 times after 

27 wailing 10 milliseconds each time. If quiesce processing can still not continue, go to TDMFMVOL. 

28 If this a Slave system that has already completed quiesce processing for this volume as indicated in 

29 MSVQMASK, then go to TDMFMVOL. If this is a Master system and the volume has completed the 

30 quiesce process, go to CHECK SYNCH COMPLETE. 

31 Issue an IOSLEVEL macro to set the IOSLEVEL to the DDR level for the source volume. Call the 

32 IOSGEN_ROUTINE. If the IOSGEN_ROUTINE indicates that the source volume is not quiesced due to 

33 either a source volume is active with a customer I/O operation or the source volume is in a reserve status, then 

34 gotoTEST_DEVICE_STATUS. 

35 Issue an IOSLEVEL macro to set the UCBLEVEL to the DDR level for the target volume. Set the TDMF I/O 

36 Monitor to "quiet mode". If this is a Slave system, set the respective system bit in MSV_QMASK. On a 

37 Master system, the quiesce complete indicator is delayed being set until TDMFMVOL processing Go to 

38 TDMFMVOL. 
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1 IOSGEN_ROTriTO£: 

2 This is a generalized routine that provides multiple functions similar to an MVS IOSGEN macro. Basically, 

3 the routine returns with a return code of zero if the source volume has no active I/O operation and there is no 

4 reserve issued against it, in a shared DASD environment. If a source volume is active with an I/O operation, 

5 a return code of four is set. If an I/O operation is not active on the source volume, but a reserve or pending 

6 reserve is outstanding, return with return code four. 

7 

8 CSECT: TDMFAVOL 

9 The purpose of this phase is to perform activate processing for a specific volume. 

10 If activate processing has not been requested via MSV_AMASK, go to TDMFNVOL. If this system has 

11 already completed activate processing for this volume as indicated in MSV_AMASK or in 

1 2 MS V„VOL_FL AG. then go to CSECT TDMFMVOL. 

13 Call LOCK_LOCAL. Call LOCKCMS. Call UCB_ALLOCATE for the source and target volumes. Call 

14 MODIFYUCBDDT to request a chain operation for the source and target volumes. Set the TDMF I/O 

15 Monitor to "active mode". Call UNLOCK_CMS. Call UNLOCK_LOCAL. 

1 6 If this is a Master system, issue ATTACHX macro to create this volume's migration copy sub-task which will 

17 control three sub-phases previously described. If this is a Slave system, indicate that activate volume 

1 8 processing is complete in MS V_AMASK for this system, and go to TDMFMVOL. If this is a Master system, 

19 set volume activate complete in MSV_VOL_FLAG and to go TDMFMVOL. 

20 UCB_ALLOCATE: 

2 1 The purpose of this phase is to prevent the MVS operating system from changing the device status of the 

22 source and target volume during a migration. This is done by updating the UCB with pseudo-allocation 

23 information. 

24 

25 CSECT: TDMFNVOL 

26 The purpose of this phase is to perform initialization processing for a specific volume. 

27 lfimtializaUonprocessmghasnc4beenrequested\iaMSV_tMASK,goto 

28 If this system has already completed initial i zation processing for this volume as indicated in MSV_IMASK, 

29 and this is a Slave system, then go to NEXT_VOLUME_PROCESSING. If this Slave system has completed 

30 volume initialization processing, set the respective system bit MSV_IMASK. On all systems, set an indicator 

31 in MS V_TDMF_ ACT_MASK that tlus TDMF sjstem is active with at least one volume migration. 

32 Increment TDMF_NUM_SYS_ACT which represents the number of systems that are active. Increment 

33 MSE_SV_NUM_ACnVE which represents the number of volumes that are active on this system. If this is 

34 a Slave system, go to NEXTJVOLUME_PROCESSING. Set the activate request function bit in 

35 MSV_AMASK. If the number of volumes is equal to one, go to TDMFAVOL, else go to 

36 NEXT_VOLUME_PROCESSING. 

37 CSECT: TDMFMVOL 

38 The TDMF I/O Monitor monitors I/O update activity against the source volume during the life of a normal 

39 migration, from the time the volume is activated until the volume has completed quiesce processing and no 
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1 further customer VO operations are allowed against the source volume. The TDMF design allows the TDMF 

2 I/O Monitor (TDMFIMON) to collect information concerning the cylinders and tracks on a volume that have 

3 been updated, and this infonnation is saved into the TDMFVRNQ control block. During TDMFUVOL 

4 processing a Slave system will write the results of its output TDMFVRNQ control block to the COMMDS in 

5 a specified location for each volume in the migration process. During TDMFCVOL processing, the Master 

6 system reads all TDMFVRNQ control blocks for all Slave system that have been previously written. 

7 The TDMF system during processing within this CSECT, must convert and merge all cylinder and track 

8 information specific to a volume, located within all TDMFVRNQ control blocks for that volume and the 

9 TDMFVRNQ control block still in memory on the Master system into a volume refresh bit map 

10 (TDMFVRBM) control block. Thus the TDMFVRBM control block represents the cumulative update 

1 1 requests that have occurred during the last TDMF processing phase. 

12 If this is a Slave system, go to SLAVE_UPDATE_PROCESSING. 

1 3 MAS TER_UPD A TEPROCES SING: 

1 4 Call EXCHANGEJVRNQ. Process all VRNQ entries located in TDMFVRNQ control block being processed, 

15 updating the active TDMFVRBM control block. Update the time stamp indicating the time of the control 

16 block that has just been processed by the Master system. If the copy sub-phase is not complete, go to 

1 7 NEXT_VOLUME_PROCES SING. 

18 If the refresh sub-phase has started and is not complete, go to NEXT_VOLUME_PROCESSING. If the 

19 refresh sub-phase has started and is now complete, go to SET_S YNCH. Else call EXCHANGE_VRBM. Call 

20 XPOST_SYNCH. 

21 SET_SYNCH: 

22 If the source volume is not quiesced on this system, go to NEXT_VGLUME_PROCESSING. Set the volume 

23 quiesced flag in MSVVOLFLAG. If the synchronization sub-phase is not waiting to be posted, go to 

24 NEXT_VOLUME_PROCESSING. Call EXCHANGEVRBM. Determine if the number of cylinders and 

25 tracks to be updated during the possible synchronization sub-phase can be accomplished within the 

26 synchronization time goal value specified by the customer Goto SYNCHVOLUMENOW. If the customer 

27 set the synchronization time goal value to 999 (as soon as possible), go to SYNCH_VOLUME_NO W. If the 

28 synchronization time goal cannot be met, go to ANOTHER_REFRESH. 

29 SVNCH_VOLUME_NOW: 

30 Indicate that this volume is ready for synchronization. If this volume is not part of a single group session and 

31 no prompt is required, then post the synchronization sub-phase to begin processing. Go to 

32 CHECK_SYNCHRONIZAHON_COMPLETE If this volume is not part of a single group session and a 

33 prompt is required, go to NORMAL JPROMPT TESTING. 

34 Since this volume is a part of a single group session, go to ANOTHER REFRESH if volumes in the session 

35 have not reached a synchronization ready state. 

36 
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1 NORMAL_PROMPT_TESTIN G: 

2 If a response has not been received from the TDMF TSO Monitor to continue synchronization, go to 

3 ANOTHERREFRESH. If a positive acknowledgment has not been received, go to ANOTHER_REFRESH. 

4 XPOSTJSYNCH: 

5 Post the synchronization sub-phase to begin synchronization processing. Go to 

6 NEXT_VQLUME_PROCESSING. 

7 ANOTHERREFRESH: 

8 Indicate that this volume requires another refresh pass Indicate that the previous refresh process has 

9 completed. Indicate that the source volume is no longer quiesced. Go to SET_RESUME in CSECT 

10 TDMFVOL. 

11 SlAVE_UPDATEJ>ROCESSING: 

12 Call EXCHANGE_VRNQ. Go to NEXT_VOLUME__PROCESSING. 

13 EXCHANGE_VRNQ: 

14 This routine exchanges the pointers between active and back-up TDMFVRNQ control blocks. This allows 

15 the TDMF I/O Monitor (TDMFTMON) to save its new TDMFVRNQ information into a newly supplied 

16 TDMFVRNQ control block. The TDMF system keeps a time stamp, by system and by volume, containing 

17 the time of the last TDMFVRNQ control block that was processed into the TDMFVRBM control block. If 

18 this time stamp is equal to the time stamp in the output TDMFVRNQ control block previously specified for 

19 output processing, this indicates that the TDMF Master system has successfully processed the output 

20 TDMFVRNQ control block into the TDMFVRBM control block. In this case, the back-up TDMFVRNQ 

2 1 control block is moved to the output TDMFVRNQ control block for subsequent processing. If the time stamps 

22 are not equal, this indicates that the TDMF Master system has not processed the information located in the 

23 TDMFVRNQ control block and that the new information located in the back-up TDMFVRNQ control block 

24 must be appended to the current output TDMFVRNQ control block, and the time stamp updated in the 

25 TDMFVRNQ control block. 

26 EXCH ANGE_VRBM : 

27 This routine exchanges the pointers between active and back-up TDMFVRBM control blocks. Additionally, 

28 the routine calculates and updates counters within the TDMFVRBM control block header indicating the 

29 number of complete cylinders and the number of individual tracks that will be either refreshed or 

30 synchronized when this TDMFVRBM is used by the TDMF copy sub-task respective's sub-phase. These 

3 1 counts in conjunction with the total I/O device service time accumulated by module TDMFSIO, when reading 

32 or writing from the migration volumes, allow the code at label SET_SYNCH above, to determine if the 

33 synchronization sub-phase can be accomplished within the synchronization time goal value specified by the 

34 customer. 

35 The TDMFVRBM control block is used as input to the refresh sub-phase and the synchronization sub-phase 

36 that execute as part of the copy sub-task. The refresh and synchronization sub-phases use the back-up 

37 TDMFVRBM control block to indicate which cylinders or tracks need to be refreshed during that sub-phase. 

38 As cylinders and tracks are refreshed, an indication is marked in a cumulative TDMFVRBM control block 

39 which indicate all cylinders or tracks that were ever refreshed during those two sub-phases. 

40 CSECT: TDMFUVOL 
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1 Set up parameters to update the COfvfMDS system record zero, which contains all information in the 

2 TDMFMSE, TDMFMSV, and TDMFMSVE control blocks. In addition, if this is a Slave system, request that 

3 the I/O operation also write all TDMFVRNQ control blocks. This I/O operation is performed with a call to 

4 routine C ALLJCOM. Upon completion, a call to RELE ASE_COM i s made in order to release control of the 

5 COMMDS for use by other systems. 

6 If additional processing is required to complete the migration of airy volume, a TDMF system (Master or 

7 Slave) will wait for a specified period of time as defined by the current systems response time. At the 

8 completion of the wait interval, go to BEGIN, which causes a re-read of the COMMDS and the beginning of 

9 the next TDMF processing cycle. If no additional processing is required for any volume migration, go to 
1 0 CSECT TDMFEVOL after setting return codes for module TDMFVOL processing. 

11 

12 CSECT: TDMFEVOL 

13 The TDMF design of the COMMDS provides a repository of all messages that may be issued from any TDMF 

14 system (Master or Slave) and any messages issued concerning an individual volumes' migration. This 

15 provides an audit trail within the TDMF system that may be used during or after completion of a TDMF 

1 6 session using the TDMF TSO Monitor. 

1 7 Set up parameters to write this systems message information and all volume migration message information 

18 located in the TDMFYMSG control blocks. Multiple calling sequences to CALLJCOM will write this 

1 9 information to the COMMDS. 

20 The TDMF design of the COMMDS provides a repository of the status of all control blocks that are used by 

2 1 this TDMF system (Master or Slave). This provides an audit trail within the TDMF system that may be used 

22 for diagnostic purposes during or after completion of a TDMF session using the TDMF TSO Monitor. 

23 Set up parameters to write this systems diagnostic information located in the TDMFWORK, TDMFVRBM, 

24 and the system TDMF control blocks. Multiple calling sequences to CALLJCOM will write this information 

25 to the COMMDS. 

26 Cal 1 MODIFYJJCBDDT with an unchain request for ail volumes involved in the migration session to ensure 

27 all modifications made to the operating system have been successfully removed. If this is a Master system, 

28 DETACH all copy sub-tasks that were created during TDMF A VOL processing. Return to the calling module, 

29 TDMFMAIN. 

30 
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1 MODULE: TDMFTMON 

2 CSECT: TDMFIMON 

3 The purpose of this module is to monitor all customer I/O operations to the source and target volumes during 

4 the life of an active migration session. This module implements the primary design objective within the 

5 TDMF system, which is to ensure that data integrity of the target volume in that any update activity by 

6 customer I/O operations that changes the data located on the source volume must be reflected to the target 

7 volume. The TDMF design allows that only volumes involved in a migration session are monitored for I/O 

8 operations allowing any I/O operations to volumes not involved within the TDMF migration session to not 

9 be impacted in any fashion. 

10 The TDMF design allows this module to receive control twice during the life of an I/O operation to a source 

11 or target volume involved in a migration session. It first receives control via the Device Dependent Start I/O 

12 Exit from MVS' IOS when an I/O operation attempts to start upon eilher the source or target volume. This 

1 3 entry point runs as an extension, effectively, of MVS ' IOS, running in Supervisor State, Protect Key Zero, and 

14 is disabled for external and I/O interrupts while the MVS local and IOS UCB locks are held. 

15 This entry point examines the starting request to determine if this request is for the source or target 

16 volume/device involved in the TDMF migration session. Module TDMFVOL, CSECT TDMFAVOL, creates 

17 a TDMFDDTE control block with it's entry address in the DDT SIO exit location pointing to the SIO main 

1 8 entry point of this module. 

19 The TDMF design demands that any errors detected or that cannot be handled by the code within this module, 

20 to notify the TDMF main task executing, that it is necessary to terminate this volumes migration due to an 

21 error to prevent loss of data integrity to the customers data being migrated to the target volume. The 

22 TDMFIMON communicates this error condition to the TDMF main task via deactivating the I/O Monitor 

23 which wili be recognized by the main TDMF task in the routine logic of label CHECK_IMON in CSECT 

24 TDMFTVOL of module TDMFVOL, and by similar logic within the TDMFCOPY module for the copy sub- 

25 task. 

26 The TDMF design requires and does not allow any customer I/O operations that would destroy information 

27 located on the target volume being migrated to are prevented. If the I/O operation is being directed to a source 

28 volume, the I/O operation must be allowed to continue without hindrance. In order to evaluate the successful 

29 completion or non-completion of an I/O operation by the customer applications, the second entry point will 

30 receive control via at this modules Disabled Interrupt Exit (DIE) address. This address is placed into the 

3 1 IOSB in lieu of any other DIE address that may exist after saving the DIE address if necessary. The DEE 

32 routine will receive control at the completion of the I/O operation on the source volume or upon receipt of an 

33 interrupt due to an intermediate request (Program Control Interrupt, PCI). 

34 The second entry point receives control as stated in the above paragraph and examines the channel program 

35 that has just completed upon the source volume It*s purpose is to create in the case of any write operation 

36 the location, cylinder and track, upon the source volume that were affected by the execution of this channel 

37 program. The information is collected in an active TDMFVRNQ control block. Each executing TDMF 

38 system will periodically exchange the pointers between an active and back-up TDMFVRNQ control block, 

39 providing an empty TDMFVRNQ control block for additional data collection by TDMFIMON for this source 

40 volumes migration. These back-up TDMFVRNQ control blocks will then be processed by the TDMF main 

41 task, module TDMFVOL, CSECT TDMFMVOL in the routine EXCHANGE_VRNQ. 

42 This TDMF design thus ensures data integrity based upon the architectural structure of channel programs and 

43 their channel command words. Therefore, the structure of files, data sets, and/or data bases are not 
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1 compromised and cannot impact the data integrity of the TDMF design. Additionally, channel programs that 

2 may be directed to a source volume that do not affect that data located upon the source volume also cannot 

3 impact the data integrity of the migrating volume. 

4 A third entry point is allocated to be used for potential future implementation of I/O operations using 

5 suspend/resume channel operations protocol. 

6 A fourth entry point is used within the TDMF system to ensure the data integrity of the COMMDS during it's 

7 reserve processing sequence. This entry point provides compatibility with MVS' IOS and physical data 

8 integrity regardless of the physical volume protection normally invoked by MVS* Global Resource 

9 Serialization (GRS) component or replacement component used by a customer, and properly manages both 

10 reserve and reserve pending conditions. 

1 1 This component executes in 3 1 -bit addressability mode and is protected by a Functional Recovery Routine 

1 2 (FRR) to ensure non-destructive termination of a volumes migration and to prevent abnormal terrmnation of 

1 3 MVS* IOS component. Additionally, the first two entry points always validate all TDMF control blocks upon 

1 4 receipt of control to ensure proper execution of the system. Any validation error of these control blocks results 

15 in the termination of the volumes* migration. 

16 STARTIOENTRYJPOINT: 

17 This entry point finds via the I/O Queue (IOQ) Element, the address of the IOSB and the address of the UCB, 

18 that this I/O request is being directed to. If the TDMFDDTE control block indicates that the TDMF I/O 

1 9 Monitor is in "quiet mode*" , this code returns to the standard MVS SIO Exit address that has been previously 

20 saved in the TDMFDDTE control block. 

21 An examination is made to determine if this I/O operation is being directed to a target volume. If so, the 

22 Channel Command Words (CCWs) in the channel program are examined to determine if the I/O operation 

23 is attempting to modify the data upon the target volum e or to read data upon the target volume other than the 

24 volume serial number. If so, this I/O operation is terminated with a special return code to MVS' IOS to 

25 indicate that this I/O operation is being prevented. If the I/O request is attempting to read the target volume 

26 serial number or is not accessing, in any fashion, any other data an the target volume , allow the I/O operation 

27 to continue normally. 

28 If the I/O operation is being directed to a source volume, the address stored in the IOSDIE field of the IOSB, 

29 if it exists, is saved in the TDMFDDTE. The address in the IOSDIE field of the IOSB is then replaced with 

30 an address to provide re-entry into the second TDMFIMON entry point. 

31 In order to allow the continuation of an I/O operation normally, control is then passed to the normal MVS 

32 DDT SIO entry point that has been previously saved. 

33 DISABLED JCV TE RRUPTJEXIT_E N TR YPO INT: 

34 This entry point is passed the address of the IOSB and the UCB address, that this I/O request is being directed 

35 to. If the TDMFDDTE control block indicates that the TDMF I/O Monitor is in "quiet mode", this code 

36 returns to the caller which is MVS 1 IOS if no IOSDIE address was saved in the TDMFDDTE control block 

37 or to the address that was saved in the TDMFDDTE control block during the previous entry point processing. 

38 If the IOSB contains a IOSDIE address that was saved, call that routine and after processing by that routine, 

39 the routine will return to this module. The IOSDIE pointer in the IOSB is normally used to transfer control 

40 to a customer supplied routine to be used in the case of a Program Controlled Interrupt (PCI). The PCI 

41 routine receiving control might dynamically modify the CCWs within the channel program based up 
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1 application I/O operational requirements. In order to ensure transparency to the customers specified PCI exit 

2 routine or DIE routine, the IOSDIE address is temporarily restored to it's original value during the life of 

3 execution of the PCI/DDE routine specified Upon return to the TDMFIMON routine, which effectively 

4 appears to the PCI/DEE routine as MVS IOS, the IOSDIE routine address in the IOSB is again restored to the 

5 address of the DI S ABLED_TNTERRUPT_EXIT_ENTRY_POINT above. This ensures in the case of a non- 

6 terminating or non-ending I/O operation that the TDMFIMON DIE entry point will again receive control 

7 during presentation of the next I/O interrupt whi ch may either be another intermediate (PCI) interrupt or the 

8 final I/O interrupt signaling completion of the channel program 

9 CCWJ5CANLOOP: 

1 0 The channel program that has received an I/O interrupt, is scanned from the first CCW that has not been 

1 1 previously scanned until the last CCW that has been executed. During this scan, each CCW operation code 

12 (op code) is used to index into a translate and test table providing multiple functions. The first is the address 

13 of a routine to receive control due to the appearance of the specified CCW op code. The second is the virtual 

14 address of the specified data operand in the CCW after being converted from a real address to a virtual 

1 5 address. The third is an indication whether this virtual address may be used by the receiving dependent op 

16 code routine. 

17 If a dependent op code routine requires that the virtual address exist, and is not available for use by the 

18 dependent op code routine, an error indication is signaled and the volume migration termination request is 

19 invoked. This may occur if the operating system pages out the MVS Page Table Entries forthe address space 

20 issuing the I/O operation. 

21 There are actually two translate and test tables associated with the channel command op code scanning 

22 mechanism One table is used for standard 388x control units and the other for 399x control units. 

23 The dependent op code routine address returned may be the address of a CCW ignore routine. Which infers 

24 that this operation code does not require further examination. An example of this would be an CCW op code 

25 for a read operation of any type, since a read op code does not affect data integrity. The dependent op code 

26 address returned may be an address of an invalid CCW op code routine which infers that this op code may 

27 not be used during the life of the migration process as it may inadvertently affect the data integrity of the 

28 volume in an unknown fashion. An example of this would be the existence of a diagnostic write CCW op 

29 code. 

30 A normal dependent op code routine will receive control and save any appropriate control information that 

3 1 may be used later in the processing of the channel program and if the I/O operation indicates that any write 

32 operation is occurring on the source volume, calls the UPDATEVRNQ routine. 

33 Upon completion of this dependent op code routine and optional call to the UPDATE VRNQ routine, the 

34 scanning mechanism continues with the next CCW to be processed. 

35 Upon completion of the scan of the channel program, return to the MVS IOS caller. 

36 UPDATE VRJSQ: 

37 Build the Volume Refresh Notification Queue (VRNQ) entries in the active TDMFVRNQ control block. Each 

38 entry consists of a sixteen-bit half-word entry indicating the affected cylinder number of the source volume 

39 that is being updated, a eight-bit (one byte) starting track number of that cylinder on the source volume thai 

40 i s being updated and an eight-bit (one byte) number of consecutive tracks that are being affected by this write 

41 operation. In order to conserve entries in the TDMFVRNQ control block, intelligence is provided to 

42 recognize the consecutive I/O operations thai might be updating adjacent tracks on the same cylinder will 
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1 attempt to use the same VRNQ entry. For example, a write I/O operation on cylinder 25, tracks 2 and 3, 

2 would create an entry of cylinder 25, starting track 2 for a length of 2 tracks. If a subsequent I/O operation 

3 updates cylinder 25, tracks 4 and 5, the previous VRNQ entry will be changed to that of cylinder 25, starting 

4 track 2 for a length of 4 tracks. 

5 There are cases when using keyed operations for searching purposes, that the system cannot determine the 

6 actual track number within that cylinder that the I/O operation may affect. In that case, a VRNQ entry 

7 encompassing the complete cylinder is created. For example, an entry of cylinder 25, startingtrack zero, with 

8 the number of tracks equal to 15, which is the maximum number of tracks per cylinder within the current 

9 DASD architecture supported. 

10 
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1 Module: TDMFCOPY 

2 CSECT: TDMFCOPY 

3 The purpose of this module is to provide the functions required by the TDMF copy sub-task, which is attached 

4 as a sub-task by module TDMFVOL, CSECT TDMFAVOL, which is created with an ATTACHX macro 

5 during its processing. This module implements three sub-task phases, the copy sub-phase, the refresh sub- 

6 phase, and the synchronization sub-phase. Additionally, a preceding sub-phase to the copy sub-phase may 

7 be executed if the customer requested the purge option when migrating from a source volume of one size to 

8 a target volume of a larger size. This module executes in 3 1-bit mode, Supervisor State, Protect Key Zero. 

9 This module contains a MVS Extended Set Task Abend Exit (ESTAE) recovery routine. This routine ensures 

1 0 that the TDMFCOPY sub-task receives control in the event that the TDMF copy sub-task attempts an abend 

1 1 condition. If any sub-phase below is posted with a termination request, the sub-phase and the copy sub-task 

12 are terminated with an error condition. 

1 3 When migrating a source volume to a larger target volume, meaning that the target volume contains more 

14 cylinders than the source volume. If the customer does not request the optional purge request, go to 

1 5 VOLUME_COPY. If the purge option is requested, prior to the copy sub-phase, each cylinder on the target 

1 6 volume is effectively erased starting with the number of cylinders existing upon the source volume plus one 

1 7 until the last cylinder of the target volume has been erased. 

18 VOLUME.COPY: 

19 This is the copy sub-phase. Prior to the read operation of each cylinder call CHECKJMON. Each cylinder 

20 on the source volume is read a cylinder at a time, and written a cylinder at a time to the target volume, starting 

2 1 with cylinder zero until the last cylinder of the source volume has been read and written to the target volume. 

22 These I/O operations are requested based upon parameters passed to module TDMFIVOL, CSECT 

23 TDMFIVOL. 

24 VOLUME.REFllESH; 

25 This is the refresh sub-phase. The back-up TDMFVRBM control block is used to indicate the cylinders and 

26 tracks that require a refresh pass. A Volume Refresh Bit Map (VRBM) entry consists of a sixteen-bit half- 

27 word field. Bit 15 of the half-word indicates that ail the information located upon that cylinder represented 

28 by this entry must be refreshed, meaning all tracks read from the source volume and written to the target 

29 volume. If bit IS is off, and any other bits (zero through 14) are equal to one, those respective bits indicate 

30 which selective tracks must be read from the designated cylinder on the source volume and copied to the 

3 1 respective cylinder on the target volume. 

32 Prior to the read operation of each cylinder, call CHECKJMON. Each cylinder on the source volume is read 

33 a cylinder as specified, and written a cylinder as specified to the target volume, starting with cylinder zero 

34 until the all VRNQ entries have been processed. These I/O operations are requested based upon parameters 

35 passed to module TDMFIVOL, CSECT TDMFIVOL. As complete cylinders and/or individual tracks are 

36 copied from the source volume to the target volume, this information is propagated to the cumulative 

37 TDMFVRBM control block, signifying that these cylinders and/or tracks have been refreshed. 

3 8 The refresh sub-phase then waits until it has been posted by the TDMF main task as to whether to continue 

39 to the synchronization sub-phase or to begin processing a subsequent VOLUMEREFRESH pass. 

40 VOLUMILSYNCHRONIZATION: 
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1 This is the synchronization sub-phase. The back-up TDMFVRBM control block is used to indicate the 

2 cylinders and tracks that require a refresh pass A Volume Refresh Bit Map (VRBM) entry consists of a 

3 sixteen-bit half-word field. Bit 15 of the half-word indicates that all the information located upon that 

4 cylinder represented by this entry must be refreshed, meaning all tracks read from the source volume and 

5 written to the target volume. If bit 15 is off, and any other bits (zero through 14) are equal to one, those 

6 respective bits indicate which selective tracks must be read from the designated cylinder on the source volume 

7 and copied to the respective cylinder on the target volume. 

8 Prior to the read operation of each cylinder, call CHECK_IMON. Each cylinder on the source volume is read 

9 a cylinder as specified, and written a cylinder as specified to the target volume, starting with cylinder zero 

1 0 until the all VRNQ entries have been processed. These I/O operations are requested based upon parameters 

1 1 passed to module TDMFIVOL, CSECT TDMFIVOL. As complete cylinders and/or individual tracks are 

12 copied from the source volume to the target volume, this information is propagated to the cumulative 

1 3 TDMFVRBM control block, signify ing that these cylinders and/or tracks have been synchronized. 

1 4 Upon completion, the ECB upon which the TDMF main task is waiting synchronization complete, is posted. 

16 VOLUME_COMPARE: 

17 This is a optional post copy sub-task function. If the customer did not request the compare option, go to 

1 8 POSTJECB. Prior to the read operation of each cylinder, call CHECKJMON. Each cylinder on the source 

1 9 volume is read a cylinder at a time, and each cylinder on the target volume is read a cylinder at a time. At 

20 the completion of these read operations, the data read from both volumes is compared, byte-by-byte, to ensure 

21 data integrity has not been compromised. This operation starts with cylinder zero until the last cylinder of 

22 the source volume has been read and compared to the target volume. These I/O operations are requested based 

23 upon parameters passed to module TDMFTVOL, CSECT TDMFIVOL. 

24 POST_ECB: 

25 Upon completion, the ECB upon which the TDMF main task is waiting synchronization complete, is posted. 
26 

27 CHECKJMON: 

28 If the main TDMF task has requested termination of the copy sub-phase, return to the caller. If the I/O 

29 Monitor is not active, return to the caller signaling that the I/O Monitor is "dead". Invoke termination of 

30 volume migration. This check is made for both the source and target volumes m the migration pair. If the 

3 1 TDMFDDTE control block is not properly chained to the MVS UCB, return the same error condition 

32 
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1 MODULE: TDMF1COM 

2 CSECT: TDMFICOM 

3 The purpose of this module is to generate channel programs consisting of Channel Command Words (CCWs) 

4 to implement I/O operations being requested to the Communications Data Set (COMMDS) via parameters 

5 passed by the caller. The caller may request that the system initialize the COMMDS, or to selectively read 

6 or write caller specified TDMF control blocks as required by the caller. This module is called from the TDMF 

7 main task only. 

8 This module contai ns a MVS Extended Set Task Abend Exit (ESTAE) recovery routine. This routine ensures 

9 that the TDMFICOM module receives control in the event that the TDMFICOM module attempts an abend 

1 0 condition. This code executes in 3 1 -bit mode, Supervisor State, Protect Key Zero and is enabled for I/O and 

1 1 external interrupts. The program generates CCWs only in format one CC W format. 

1 2 The TDMF design provides that all control blocks arc 4K in length or a logical multiple of 4K in length, or 

1 3 that multiple control blocks may exist evenly into one 4K page. Although the COMMDS may reside upon 

14 either a 3380 or 3390 device type track, in order to provide compatibility, only ten data records, each 4K in 

15 length, are used on any track within the COMMDS to contain the TDMF control blocks. During the 

1 6 initialization process of the COMMDS, if the tracks are allocated on a 3390 device type, two additional data 

1 7 records are written on each track containing dummy records. Due to the design of the TDMF COMMDS, 

1 8 multiple write operations from multiple TDMF systems may be executed simultaneously to all cylinders within 

19 the COMMDS without reserve/release protections, except logical cylinder zero of the COMMDS. This is due 

20 to each TDMF system is allocated a cylinder into which the respective information about that system may be 

2 1 written without regard to another system writing to the same cylinder. In other words, the Master system 

22 (system number one) always writes its data onto relative cylinder one. The first Slave system (system number 

23 two) writes its data on logical cylinder two, and so on. 

24 The format of each cylinder allocated to a system, provides up to 64 TDMF VMS G control blocks to be written 

25 across tracks zero through six of the specified cylinder. The last six records of track six on the specified 

26 cylinder will remain as dummy formatted records. The TDMFWORK control block and its imbedded source 

27 and target TDMFDDTE entries, each currently only require 2K. Therefore, two TDMFWORK/TOMFDDTE 

28 control blocks may be written within one 4K page. Tracks seven through ten of the specified cylinder will 

29 contain all TD MFWORK/TDMFDDTE control block information for up to 64 volumes. The last eight 

30 records of track ten on the specified cylinder will remain as dummy formatted records. Track eleven on the 

3 1 specified cylinder will contain all records related to the main TDMF control block for the specified system. 

32 This contains information including system messages, messages that may have been received from the TDMF 

33 TSO Monitor from the TDMFTSOM control block, plus information from the TDMF authorization checking 

34 mechanisms. This information currently requires four records and thus, the additional six records will remain 

35 as dummy formatted records. Tracks twelve through fourteen currently contain only formatted dummy 

36 records and are availabl e for future expansion. 

37 After all cylinders allocated to systems in the TDMF session are formatted, the rest of the extent allocated to 

38 the COMMDS is formatted with dummy records. The first of these cylinders is later used to contain the 

39 cumulative TDMFVRBM control block representing the map of all cylinders and tracks that were updated 

40 during the refresh and synchronization sub-phases during the volumes migration. 

4 1 Logical cylinder zero contains what is referred to as System Record Zero on tracks zero and one of that logical 

42 cylinder. System Record Zero consists of 20 4K pages containing all the information in the TDMFMSE, 

43 TDMFMSV, and TDMFVSVE control blocks, collectively. Relative tracks two through 14 on logical cylinder 
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1 zero contain specified locations to hold the TDMFVRNQ control blocks required by the TDMF session. Since 

2 only ten records per track are allowed to contain data, there are thirteen tracks that may contain all possible 

3 TDMFVRNQ control blocks within the TDMF session as currently implemented. 

4 An artificial implementation restriction has been placed into the TDMF design to limit the number of systems 

5 involved in a TDMF migration session times the number of volumes within the migration session may not 

6 exceed 128. This architectural implementation restriction could and may be removed in future releases of 

7 TDMF as required by using additional cylinders to contain the TDMFVRNQ control blocks. In order to limit 

8 the impact upon I/O operations to the COMMDS during the initial releases of TDMF, it was felt that this 

9 artificial implementation restriction would be of a minor impact to customer requirements. 

1 0 Architecturally, the TDMF system was designed to support 32 operating systems and 64 volumes within one 

1 1 TDMF session. 

12 All I/O operations to the COMMDS represented by an IOSB contain an lOSLEVEL of one, indicating normal 

13 priority, and that miscellaneous driver is being used and that the IOSB is using command chaining, the 

1 4 channel program is complete, and that the MVS Input/Output Supervisor should by-pass channel program 

1 5 prefixing. This prevents any manipulation of the constructed channel programs by MVS* IOS component 

1 6 INITIALIZE_COMMDS: 

1 7 This routine is used to initialize the COMMDS and to generate channel programs consisting of Channel 

1 8 Command Words (CCWs). It may only be called by the TDMF Master system. It formats the complete extent 

1 9 upon the DASD volume allocated to the COMMDS with dummy records and then re-writes system record 

20 zero. 

21 Additionally, during formatting, the information iu the TDMFMSE, TDMFMSV, TFMFMSVE, and 

22 TDMFWORK control blocks, is updated to contain the assigned cylinder, track and record locations. 

23 During formatting the information within the COMMDS is protected via a hardware reserve CCW to prevent 

24 access from another system until the formatting is complete. A routine is called to build the Service Request 

25 Block (SRB) and the Input/Output Supervisor control Block (IOSB) which represent each I/O request during 

26 the initialization process. This SRB/IOSB pair is passed to the TDMFSIO module to actually issue the I/O 

27 request and awail its completion by the operating system Upon completion of the I/O operation, control is 

28 returned to the calling module. 

29 Call module TDMFAUTH for verification of authorized CPU environment 

30 REA»_COMMDS: 

31 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

32 selectively read user specified TDMF control blocks or a combination thereof, from the COMMDS. 

33 Additionally, if the user requests information located in system record zero, the system provides a hardware 

34 reserve CCW to prevent access from another system until the update of system record zero is subsequently 

35 requested and a release operation is performed. If the reserve CCW has been added, the subsequent IOSB 

36 below, will contain the DIE address located in module TDMFIMON's fourth entry point. 

37 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

38 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 

39 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

40 Upon completion of the I/O operation, control is returned to the calling module. 
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1 If this is the first read of the COMMDS by a Slave system, call module TDMFAUTH for verification of 

2 authorized CPU environment 

3 UPDATE.COMMDS: 

4 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

5 selectively update/write user specified TDMF control blocks or a combination thereof; to the COMMDS. 

6 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

7 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 

8 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

9 Upon completion of the I/O operation, control is returned to the calling module. 

10 
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1 Module: TPMFIVOL 

2 CSECT: TDMFIVOL 

3 The purpose of thi s module is to generate channel programs consisting of Channel Command Words (CC Ws) 

4 to implement I/O operations being requested to volumes being migrated by the TDMF session via parameters 

5 passed by the caller. The caller may request that the system: 

6 - read all tracks of data from the specified volume on a cylinder, 

7 - read a single track of data from the specified volume on a specified cylinder, 

S - selectively read non-adjacent tracks of data from the specified volume on a specified cylinder; 

9 - write all tracks of data from the specified volume on a cylinder, 

1 0 - write a single track of data from the specified volume on a specified cylinder, 

11 - selectively write non-adjacent tracks of data from the specified volume on a specified cylinder, 

12 - provide miscellaneous I/O functions as specified by the caller. 

13 Miscellaneous I/O functions include items such as, Read Device Characteristics information, Sense ID 

14 i nformation. Sense Subsystem Status information, Read Configuration Data information, Read Volume Label 

1 5 information, Write Volume information, reserve the volume, and release the volume. 

1 6 This module may be called from both the TDMF main task or a TDMF copy sub-task. 

] 7 This module contains a MVS Extended Set Task Abend Exit (EST AE) recovery routine. This routine ensures 

18 that the TDMFIVOL module receives control in the event that the TDMFIVOL module attempts an abend 

1 9 condition . This code executes in 3 1 -bit mode. Supervisor State, Protect Key Zero and is enabled for I/O and 

20 external interrupts. The program generates CCWs only in format one CC W format. 

21 READSOURCECYL: 

22 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

23 selectively read all tracks upon the caller specified cylinder from the user specified volume. 

24 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

25 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 

26 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

27 Upon completion of the I/O operation, control is returned to the calling module. 

28 READ_SOURCE_TRK: 

29 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

30 selectively read the caller specified track from the user specified volume. 

31 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

32 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 

33 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

34 Upon completion of the I/O operation, control is returned to the calling module. 

35 All I/O operations to a requested volume are represented by an IOSB, which contains an IOSLE VEL of the 

36 specified IOSLEVEL of Dvnaraic Device Reconfiguration (DDR), and that a miscellaneous driver is being 

37 used and that the IOSB is using command chaining, the channel program is complete, and that the MVS 

Attorney Docket No.: AMDH8132DEL Express Mail Label: TB597450458US 

\\Maunakea\sf\del\wp\amdh8\8 1 32\pa-replace.0CT03 "20-cmp2 .wpd 3/20/00-17:23 

64 



87 



6,145,066 



88 



LISTING 1 

TDMF LOGIC ® 1998 AMDAHL CORPORATION 

1 Input/Output Supervisor should by-pass channel program prefixing. This prevents any manipulation of the 

2 constructed channel programs by MVS' IOS component 

3 The setting of the IOSLEVEL to that used by DDR ensures that I/O operations may be requested against a 

4 migrating volume and even if the volume is logically quiesced. 

5 READJSOURCESEL: 

6 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

7 selectively read specific tracks from the caller specified cylinder from the user specified volume. Multiple 

8 tracks may be read with one request and are not required to be adjacent 

9 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

1 0 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 

11 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

12 Upon completion of the I/O operation, control is returned to the calling module. 

13 WRITETARGETCYL: 

14 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

15 selectively write all tracks upon the caller specified cylinder from the user specified volume 

1 6 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

1 7 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 

18 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

1 9 Upon completion of the I/O operation, control is returned to the calling module. 

20 WRITE_TARGET_TRK: 

21 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

22 selectively write the caller specified track from the user specified volume. 

23 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

24 (IOSB) which represent each I/O request during the imlialization process. This SRB/IOSB pair is passed to 

25 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

26 Upon completion of the I/O operation, control is returned to the calling module. 

27 WRITETARGETSEL: 

28 This routine is used to generate a channel program consisting of Channel Command Words (CCWs) and to 

29 selectively write specific tracks from the caller specified cylinder from the user specified volume. Multiple 

30 tracks may be written with one request and are not required to be adjacent. 

31 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

32 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 

33 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

34 Upon completion of the I/O operation, control is returned to the calling module. 

35 MISCCCW: 

36 This routine is used to generate a channel program consisting of a Channel Command Word (CCW) to 

37 perform the caller specified I/O operation. 

38 A routine is called to build the Service Request Block (SRB) and the Input/Output Supervisor control Block 

39 (IOSB) which represent each I/O request during the initialization process. This SRB/IOSB pair is passed to 
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1 the TDMFSIO module to actually issue the I/O request and await its completion by the operating system. 

2 Upon completion of the I/O operation, control is returned to the calling module. 

3 
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1 Module: TDMFSIO 

2 CSECT: TDMFSIO 

3 The purpose of this module is to issue a request to MVS* Input/Output Supervisor (IOS) component to 

4 perform the I/O operation represented by the Input/Output Supervisor control Block (IOSB) in conjunction 

5 with its Service Request Block (SRB) as requested by the caller. This module may be called from module 

6 TDMFICOM and module TDMFIVOL. Upon completion of the I/O operation requested, control will be 

7 returned to the calling module. 

8 This module executes in 31 -bit mode, Supervisor State, Protect Key Zero. This module contains a MVS 

9 Extended Set Task Abend Exit (ESTAE) recovery routine. This routine ensures that the TDMFSIO module 

1 0 receives control in the event that the TDMFSIO module attempts an abend condition. 

1 1 Before passing the I/O request to the IOS component, the module ensures that the device containing the 

1 2 volume to which the I/O operation is to be directed, is online and has available channel paths. If not, an error 

1 3 indication is returned to the caller. Since the TDMF design requires, in some cases, that I/O operations be 

1 4 attempted to a device and/or volume while I/O operations are logically quiesced, the normal MVS STARTIO 

15 macro cannot be used. This is because the MVS STARTIO compatibility interface routine, module 

16 EECVSTIO, will reset the IOSLEVEL in the IOSB to a value that may be incompatible to successftiily 

17 complete the I/O operation. Therefore, the TDMF design requires an implementation that calls module 

18 IOSVSSEQ directly, so that the I/O operation may be completed even if the device and/or volume is 

19 considered quiesced. 

20 Upon completion of the I/O operation requested, the elapsed time of the I/O operation is calculated, which 

2 1 provides device service time information. This device service time information is required for use by the 

22 TDMF TSO Monitor and the routine called EXCHANGEJVRBM in CSECT TDMFMVOL of module 

23 TDMFVOL. 

24 The I/O operation is checked for any error conditions upon its completion and an indication returned to the 

25 caller of the success or failure of the I/O operation. 

26 The module contains a normal and abnormal channel end appendage routines and a post routine to be used 

27 as routines called by MVS' IOS component. 
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1 Transparent Data Migration Facility TSO Monitor System 
2 

3 REXX EXEC: TDMF 

4 The initial REXX EXEC sets up an environment under Interactive Structure Program Facility (ISPF), 

5 allocates all library definitions required for use for the TDMF TSO Monitor and goes to REXX EXEC 

6 TDMFMON. 

7 

8 REXX EXEC: TDMFMON 

9 Displays the beginning panels and upon ENTER command, processes the options as specified below. 

10 OPTION 1: 

1 1 Enters standard ISPF EDIT mode, allowing the customer to edit the supplied sample TDMF JCL library and 

12 submit TDMF jobs to the MVS operating system Job Entry Subsystem (JES) 

13 OPTION 2: 

1 4 Calls module TDMFWTCH which displays pertinent information for all volume migration status within all 

1 5 TDMF sessions that are currently active. 

16 OPTION 3: 

17 Calls module TDMFMSGS which displays all messages for all volume migrations within the customer 

1 8 specified active TDMF session. 

19 OPTION 4: 

20 Calls module TDMFPERF which displays performance information for all volume migrations within the 

2 1 customer specified active TDMF session. This information can also be displayed for a TDMF session has 

22 previously completed. 

23 OPTIONS: 

24 Calls module TDMFVPST which displays all messages for all volume migrations within the customer 

25 specified previously completed TDMF session. 

26 OPTION 6: 

27 Implements options to provide diagnostic and support information. 

28 OPTION 6.1: 

29 Calls module TDMFSTAT which displays general information about an active TDMF session and its use of 

30 the COMMDS. 

31 OPTION 6.2: 

32 Displays the memory location for the length specified as accomplished as a part of the TDMFMON REXX 

33 EXEC. 

34 OPTION 6.3: 

35 Calls module TDMFCOMF which displays the control blocks requested by control block name. 
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1 OPTION 6.4: 

2 Calls module TDMFCNAV which displays specified cylinder, track and record from the requested COMMD S. 

3 OPTION 7: 

4 Calls module TDMFTMON which provides an operator interface mechanism that may be used to either 

5 terminate an active volume migration, respond to the synchronization prompt, and modify the synchronization 

6 goal time value, in seconds, dynamically. 

7 OPTION 8: 

8 Calls module TDMFUNIQ which displays customer and site information with the additional capability to 

9 dynamically add or delete authorization keys, which allows TDMF to execute upon an authorized CPU 

10 processor. 

1 1 Module; TDMFSECR 

12 Initializes the authorization keys within the specified library, which allows TDMF to execute upon an 

1 3 authorized CPU processor. 

14 Module: TDMFAUTH 

1 5 This module provides the actual authority checking to determine if the CPU is authorized to execute TDMF. This 

1 6 module can be called by modules TDMFSECR, TDMFUNIQ, or TDMFICOM, executing as a part of the main task 

1 7 during TDMF execution. 
18 

While the invention has been particularly shown and described with reference to preferred 
embodiments thereof it will be understood by those skilled in the art that various changes in form 
and details may be made therein without departing from the spirit and scope of the invention. 
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